Load code to certain regions of the flash

Rafael Pires
Posts: 1
Joined: Mon Aug 07, 2017 3:45 pm

Load code to certain regions of the flash

Postby Rafael Pires » Mon Aug 07, 2017 4:09 pm

In my application I want load some drivers to a region of the flash (specified by a custom partition table), and use those drivers on my app. Later i want be able to build my application and use those drivers without have to flash that section of the flash again. Basically i want to have pre-compiled functions stored on the flash and have those functions available on the IDF to build my own application. How can i load some parts of the code to a specific flash region? And, how can i have those functions available on the composer?

Best Regards,

Rafael Pires

User avatar
kolban
Posts: 1087
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: Load code to certain regions of the flash

Postby kolban » Wed Aug 09, 2017 3:35 pm

Howdy Rafael,
I don't have any technical answers to your questions but I am interested in the back story. What problem/puzzle/solution does having "preloaded" libraries of function solve/resolve?

My best guess is that you want to modularize an Over The Air (OTA) refresh of an application so that only portions of the application that are likely to be changed need be changed. That is only a guess though and I'd love to hear the design thinking and goal you have in mind.
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Sprite
Posts: 1231
Joined: Thu Nov 26, 2015 4:08 am

Re: Load code to certain regions of the flash

Postby ESP_Sprite » Wed Aug 09, 2017 5:35 pm

I would also be very interested in the purpose of this... For the 'how', it is possible in multiple ways. The easiest way may be to write your code, link it using a custom linker script (that makes it end up somewhere in the ESP32 address space) and end up with a binary blob with all the functions in it. Then use a script to convert the symbols in it into a linker file (for example, my elf_to_ld.sh hack is still in esp-idf/components/esp32/ld, you could use that.) Then all you would need to do is use the MMU to map the flash segment into the address space at the address specified in your linker script. (Unfortunately, esp-idf doesn't have an easy option to do this at the moment - if needed, please add a github issue for it) and then you can link your application against the linker script containing the symbols.

There are more complex options - something like a semi-dynamic loader comes to mind - bit as far as I know this is the easiest.

WiFive
Posts: 1219
Joined: Tue Dec 01, 2015 7:35 am

Re: Load code to certain regions of the flash

Postby WiFive » Wed Aug 09, 2017 10:20 pm

ESP_Sprite wrote:Then all you would need to do is use the MMU to map the flash segment into the address space at the address specified in your linker script.

What about data/bss/IRAM?

ESP_Sprite
Posts: 1231
Joined: Thu Nov 26, 2015 4:08 am

Re: Load code to certain regions of the flash

Postby ESP_Sprite » Sat Aug 12, 2017 1:52 pm

Ah - fair point, I neglected any static variables. Hmm, that's a hard one, you'd have to modify the esp-idf to steer clear of those regions as well.

WiFive
Posts: 1219
Joined: Tue Dec 01, 2015 7:35 am

Re: Load code to certain regions of the flash

Postby WiFive » Sat Aug 12, 2017 2:16 pm

ESP_Sprite wrote:Ah - fair point, I neglected any static variables. Hmm, that's a hard one, you'd have to modify the esp-idf to steer clear of those regions as well.


...and you'd have to modify the bootloader to load those sections too.

ESP_Sprite
Posts: 1231
Joined: Thu Nov 26, 2015 4:08 am

Re: Load code to certain regions of the flash

Postby ESP_Sprite » Sat Aug 19, 2017 8:54 am

FWIW, I've been looking at this a little longer. In essence, normal OSses also have this problem because a program in e.g. Linux has to be able to load multiple shared libraries per process, so the shared libraries can't use fixed addresses for their static/global variables. That is solved in one of two ways: either the dynamic linker 'fixes up' the shared library by essentially re-writing the machine code to adjust the variable addresses. The other way is to write position-independent code (PIC). (Assuming 'code in a certain region in flash' is not supposed to be pre-linked to the eventual executable, what you actually want is a sort-of shared object/library in flash.)

Given that we want to execute the code from flash (and not load it into IRAM first), only the second option remains: we don't want to modify and re-flash the shared object every time. The Xtensa compiler seems to support PIC just fine. Unfortunately, the executable generated still needs fixup when we use globals: the addresses to the globals are stored in a few bytes before each function (essentially, they are literals in xtensa-speak) and my guess is that it's up to the dynamic linker to fix these up to point to the actual data segment.

So all in all: I do think you can compile a bunch of code (with -fPIC), eke out the .text region using objcopy or a well-chosen linker script and dump it into flash; then in the ESP32 app memmap the flash pages into IRAM and call the functions you've defined there, optionally using something like a pointer table in the object. Globals and static variables seem impossible without copying the code to RAM, however. It's not impossible to code without these, though: just use the classic 'object-oriented' way of passing a 'this' struct with all globals in it to every function that needs it.

ESP_igrr
Posts: 870
Joined: Tue Dec 01, 2015 8:37 am

Re: Load code to certain regions of the flash

Postby ESP_igrr » Sat Aug 19, 2017 12:04 pm

I think if we can get something similar to ARM's -mno-data-is-text-relative (which makes static data addressable via GOT) into Xtensa target, that would be enough to have dynamic loading with no caveats.
Currently the obvious (but not nice for any non-trivial code) workaround is to *not* use static variables in the loadable module (only global/common). Then you get GOT entries for that data, which can be filled in by the loader.

ESP_Sprite
Posts: 1231
Joined: Thu Nov 26, 2015 4:08 am

Re: Load code to certain regions of the flash

Postby ESP_Sprite » Sun Aug 20, 2017 4:07 am

Well, it's going to be hairy at any rate. It's essentially a catch-22: because you don't know where your static variables live, you need a pointer to these in a known location, but because the only known locations can be relative to your code, they have to be in read-only memory. The options I see are:
[*] Go the X86_64 way, that is using a dedicated register for the static variable location. Apart from the fact that this needs a compiler modification, this is pretty hairy because the windowing ABI makes it hard to have a constant register; they rotate every time a subroutine is called.
[*]Go the uboot way and use a global (well, thread-local) variable; maybe use one of the MISC* processor registers for this. Can be done, but needs a compiler change as well.
[*]Only load libraries into IRAM. That way, we can fixup stuff in memory.

Who is online

Users browsing this forum: No registered users and 15 guests