Question: Assembler messages memory exhausted

[Mosbeard]

Hello,

i am trying to implement an OPC UA-Server on ESP32.
Here some info about my environment:
## Environment
- Module or chip used: ESP32-WROOM-32U
- IDF version : v4.1-dev-437-gd2ad0f077-dirty
- Build System: ninja|CMake|idf.py
- Compiler version (run ``xtensa-esp32-elf-gcc --version`` to find it): 8.2.0
- Operating System: Windows 10
- (Windows only) environment type: ESP Command Prompt
- Using an IDE?: No
- Power Supply: external 3.3V

Attached you find the output from idf.py build command. What does this exactly mean and what can i do? Are there some tricks to get more memory? I can't find a documentation for as.exe.
Does ESP32-WROOM-32U have too little flash?

[ESP_Sprite]

It looks like you ran out of memory on the computer esp-idf is running at; it doesn't have anything to do with any memory on the ESP32 side.

[Mosbeard]

Okay, thanks for the hint. Attached you find the c source file I want to add to my esp-idf project. When I add this I get the memory exhausted error.
Now I watched the task manager while I run the ninja command in the build directory of my project with this results:
First cc1.exe is starting, raises up to about 900 MB and then it terminates.
After that as.exe is starting, raises up to about 1800 MB and then it terminates with memory exhausted error.
Are there any memory limits for this program or esp-idf which I can configure?
When this is not possible, what else can I try to compile this c source file in my project?
What is the memory limit of as.exe dependent on?

[ESP_Angus]

Code: Select all

$ ls -lh open62541.c
-rw-r--r-- 1 gus gus 9.3M Feb 13 08:28 open62541.c
$ wc -l open62541.c
180301 open62541.c

Wow, 9.3 Megabytes and 180,000 lines. That is a large source file!

I notice that the comment at the top says "THIS IS A SINGLE-FILE DISTRIBUTION CONCATENATED FROM THE OPEN62541 SOURCE". You could perhaps try obtaining the non-single-file version and compile each file individually? That would probably solve the problem.

Are there any memory limits for this program or esp-idf which I can configure?
When this is not possible, what else can I try to compile this c source file in my project?
What is the memory limit of as.exe dependent on?

Fairly sure the only hard limit is the amount of RAM in your computer. You could try exiting other programs running in Windows, buying more RAM, or increasing the Windows page file size.

However I think the best option would be to seek out the not-single-file version of this project.

EDIT: My colleague reminded me the Windows toolchains are compiled as 32-bit binaries, so there is a limit of 2GB of RAM per executable. This is probably the limit reached by as.exe. There is apparently a "patcher" tool which can patch 32-bit Windows binaries so they can access 4GB of RAM when run under 64-bit Windows, maybe this would help. But finding smaller source files is probably the best solution!

[Mosbeard]

@ESP_Angus thank you for the information. That was very helpful.

I followed your suggestion and compiled the single files. This works until linking step(see attachment line 829: iram0_2_seg is not big enough; it would be helpful if you could explain the error message in your words),
The biggest file of the single files is the namespace0_generated.c file with about 7 MB (see attachment), so not much less than the whole amalgamated file. The size of this file is variable and dependent on the information model of OPC UA Server.
This is the library we try to implement https://open62541.org/doc/1.0/toc.html to run an OPC UA Server on ESP32-WROVER-IB module (16MB Flash).
The OPC UA Server creates an information model at start up with namespace0_generated.c. The information model consists of nodes. So the more nodes it have the bigger gets namespace0_generated.c file.

Is it possible to place such a big file on a code segment of ESP32 and how do i do this with .lf files?
Or what else can I try to implement this?

[ESP_Angus]

iram0_2_seg is not big enough;

There is a limit of 3.3MB of executable code per app. This is the most that can be mapped by the flash cache into the address space, at one time (iram0_2_seg is the non-intuitive name for the flash cache mapped region).

If the linked binary size comes out at more than 3.3MB, you'll get this error. (Note that the size of the .c or .o/.obj files doesn't always predict the final binary size because not everything gets linked in. But the error message should indicate how many bytes it is "too big" by.)

Suggest looking at ways to reduce the compiled binary size, or exclude features at link time. Are there any compilation flags you can push into the library to disable unused features? If you haven't chosen it already, go to the IDF project's configuration menu compiler settings and choose Optimize for Size, you can also set Assertions to Silent on the same display.

(This likely won't be enough by itself, but it will help.)

[Mosbeard]

Thanks for the quick and informative answer. I appreciate that. After countless attempts to downsize the library, it has not been possible to build the project. Optimize for Size was already selected and the log and debug outputs deactivated.
After studying the TRM, I understood that the 3.3 MB correspond to VAddr1. According to the TRM, it is possible to load either VAddr1 (3.3 MB), VAddr2 (4 MB) or VAddr3 (4 MB) into the address space, but it is not supported by the esp-idf. Therefore I could imagine that it should be possible to create an app that is larger than 3.3 MB and whose code is distributed to VAddr1, VAddr2 and VAddr3 and the MMU then loads the required area into the address space. But after reading the issues on github, I'm not sure what the current status is:
https://github.com/espressif/esp-idf/issues/1667
https://github.com/espressif/esp-idf/issues/1184
What is your current opinion on the topic? Is this now possible or not? If so, is there a project example?
Either way, we will have to try out other OPC UA libraries that will hopefully be suitable for the 3.3 MB IROM limit.

[Mosbeard]

Hello @ESP_Angus,

please give me a short information about this topic.

[ESP_Angus]

Hi Mosbeard,

Sorry I didn't see your reply earlier.

As per the issues you linked above, it's not currently possible to build an app that links code into the other VADDR regions. The limit is 3.3MB for IROM.

Possibly this support could be added to ESP-IDF, but I'm afraid it's not available now. If you subscribe to issue 1667 on GitHub then you'll be updated if there's any change in this support.

Angus

[Mosbeard]

Allright, thank you for the answer @ESP_Angus