Experiement to load/execute compiled C code into RAM

jcsbanks
Posts: 287
Joined: Tue Mar 28, 2017 8:03 pm

Experiement to load/execute compiled C code into RAM

Postby jcsbanks » Mon Apr 08, 2019 11:45 am

I could malloc for then load a tiny binary into RAM and define a function pointer and execute the function.

The C code uses no standard libraries or esp-idf calls, no heap, does not rely on .bss to be initialized, it doesn't even have any includes, it is just computational taking an unsigned int and returning an unsigned int, but since I have the code in C I don't want to convert it to Lua or Javascript if I can do this and I don't want to OTA update the whole binary just to add a 300 byte function. I am used to using C to embed new code in existing binaries on other microcontrollers like this, but without IDA or a simulator for Xtensa I'm a little limited.

Code: Select all

xtensa-esp32-elf-gcc -std=c99 -Os -nostdlib -c test.c
This produces test.o

Code: Select all

xtensa-esp32-elf-objcopy -O binary test.o test.bin
This produces test.bin

Further considerations:
1. Entry point to the single compiled function relative to the start of the bin. I would like it at the start, but the .rodata presently is
2. .rodata would require 32bit alignment if in IRAM unless there is a way to access the same memory as DRAM with an offset
3. Cannot load the bin into IDA to disassemble it to double check what it is doing. There is a ref to memcpy that is not in my code, want to make sure it does not end up in the final bin.

Comments appreciated. Perhaps I should just embed Lua, but I couldn't find any working examples to do that inside an ESP-IDF app and didn't really want to learn Lua. Duktape seems interesting but bulky, and mJS/Mongoose licensing is not ideal for a commercial project where we presently have no money :)

jcsbanks
Posts: 287
Joined: Tue Mar 28, 2017 8:03 pm

Re: Experiement to load/execute compiled C code into RAM

Postby jcsbanks » Mon Apr 08, 2019 12:55 pm

When I remove things that would use .bss or .rodata, the memcpy is removed and the build can actually be linked with an empty linker script and emit a map file. Would be nice to be able to use .rodata though.
Last edited by jcsbanks on Mon Apr 08, 2019 1:22 pm, edited 1 time in total.

WiFive
Posts: 2791
Joined: Tue Dec 01, 2015 7:35 am

Re: Experiement to load/execute compiled C code into RAM

Postby WiFive » Mon Apr 08, 2019 1:07 pm


jcsbanks
Posts: 287
Joined: Tue Mar 28, 2017 8:03 pm

Re: Experiement to load/execute compiled C code into RAM

Postby jcsbanks » Mon Apr 08, 2019 10:38 pm

heap_caps_malloc(size, MALLOC_CAP_EXEC | MALLOC_CAP_8BIT) fails. Reading the docs suggests that D/IRAM can do data and instructions, but it doesn't say I'm restricted to 32 bit alignment, but can find no explicit way to malloc D/IRAM or may have misunderstood the capabilities. I copied it from DRAM to IRAM but so far whilst it does not crash running the code it does not give expected results just returning a constant so need to investigate more.

ESP_Angus
Posts: 1875
Joined: Sun May 08, 2016 4:11 am

Re: Experiement to load/execute compiled C code into RAM

Postby ESP_Angus » Tue Apr 09, 2019 12:35 am

jcsbanks wrote:
Mon Apr 08, 2019 10:38 pm
heap_caps_malloc(size, MALLOC_CAP_EXEC | MALLOC_CAP_8BIT) fails. Reading the docs suggests that D/IRAM can do data and instructions, but it doesn't say I'm restricted to 32 bit alignment,
The D/IRAM is some physical RAM which is available on both the executable and data buses, so there is one address on each bus which can map to the same physical RAM. The idea behind the "capabilities based" allocator is to abstract this way, so that you either ask for executable RAM (which has an address on the IRAM bus and requires 4 byte aligned access) or data RAM (which has an address on the data bus), depending on what you need.

Suggest doing heap_caps_malloc(size, MALLOC_CAP_EXEC) and then memcpy() a 4 byte aligned size into this buffer (round up the length of the .bin file if it's not already 4 byte aligned).

I'm not 100% sure that what you're doing will work: taking unlinked object data like this and treating it as a raw executable binary. I guess it probably will work provided you don't use .rodata or any other symbols. You can use "xtensa-esp32-elf-objdump -d test.o" to check that the object file only uses a single .text section and doesn't contain any literal sections (literal sections will have placeholder data waiting to be filled in at link time, and as you don't run the linker there's no way to support these.)

jcsbanks
Posts: 287
Joined: Tue Mar 28, 2017 8:03 pm

Re: Experiement to load/execute compiled C code into RAM

Postby jcsbanks » Tue Apr 09, 2019 9:37 am

Thanks. I was just using a for loop instead of memcpy, so I had explicit control over alignment, but it seems to work if I use small constants so there is no literal section: the function runs and returns the expected value (with values being passed in and out of the function). It looks like the literal section is used for constants larger than 12 bit since a 24 bit instruction cannot hold a 32 bit value. I'm not used to literal sections so working on understanding them.

Whilst I can do without .rodata and .bss with what feels like less hassle than using a scripting language (and I guess I could add them), I do need what I think the literal section offers to have to handle it.

jcsbanks
Posts: 287
Joined: Tue Mar 28, 2017 8:03 pm

Re: Experiement to load/execute compiled C code into RAM

Postby jcsbanks » Tue Apr 09, 2019 11:06 am

It works by loading .literal and .text as a blob and then a function pointer which has an offset into the blob where .text is.

Linker script:

Code: Select all

SECTIONS
{
	.literal : {*(.literal)}
	.text : {*(.text)}
}
This is obviously a minimal method with no error checking or any concerns whatsoever!

Who is online

Users browsing this forum: Baidu [Spider], MSN [Bot] and 21 guests