Page 1 of 1
Finding a method for storing a large data before run time
Posted: Sat Aug 16, 2025 4:03 am
by leanh1234
Hi guys!
I am trying to find a method to store a large hex array in flash before run time. I also want to move that data to PSRAM in ESP32S3. Do you have any solution for storing the hex array in flash first?
Thank you very much. I hope that I will get any advice from you guys!
Best regards,
Ant Leo
Re: Finding a method for storing a large data before run time
Posted: Sat Aug 16, 2025 4:49 am
by vvb333007
You have to flash your ESP32S3 with a firmware, which creates required huge hex files on the flash using fopen, fwrite and fclose. Then you can flash your real firmware which will load that data from flash memory to the PSRAM.
Also take look at mmap functions of ESP-IDF: they allow for memory mapping of a flash partition , so you can access your flash partition as an array, without need to move your data to the PSRAM
Re: Finding a method for storing a large data before run time
Posted: Sat Aug 16, 2025 9:50 am
by MicroController
You can create a
partition for your data which can be flashed independently of your application. Using the
partitions API, your application can then copy the data from the partition in flash to PSRAM.
Alternatively, you can
embed binary data in the application.
Re: Finding a method for storing a large data before run time
Posted: Mon Aug 18, 2025 11:08 am
by leanh1234
Thanks for your suggestion! It is very informative to me. I also wonder to write the data to a segment in partition in the flash time because I saw that almost skainet examples have 2 times, one is for the main program, the other is for the model such as multinet, vad, afe. If it is imposible, how do I modify my project? I still hope get the advice from you guys! Much Love!
Re: Finding a method for storing a large data before run time
Posted: Mon Aug 18, 2025 12:21 pm
by MicroController
Sorry, I don't understand what you wrote.
You create your own partition by
creating your own partition table .csv file. The easiest way to do that is probably to take one of the default partition tables from the documentation and modify it. In that .csv file, you have to add a line with the desired name of the partition, its type (e.g. "data", or your own custom type in the range 0x40-0xFE), possibly a subtype, and the size of the partition.
You then select your "Custom partition table CSV" via menuconfig and build+flash your project.
Re: Finding a method for storing a large data before run time
Posted: Mon Aug 18, 2025 6:15 pm
by thefury
Easiest thing to do is probably store the data array in flash .rodata (with the keyword const), and have a data array in PSRAM (EXT_RAM_NOINIT_ATTR) and copy it yourself at startup. There is no way to have it transparently store the data in flash and then copy to PSRAM at startup, although that would be neato.
Code: Select all
#include "esp_attr.h"
#define BIG_DATA_BUFFER_SIZE (69420)
static const uint8_t large_data[BIG_DATA_BUFFER_SIZE] = {
// your data here
};
static EXT_RAM_NOINIT_ATTR uint8_t large_data_in_psram[BIG_DATA_BUFFER_SIZE];
// Call this at startup
void copy_to_psram()
{
memcpy(large_data_in_psram, large_data, sizeof(large_data));
}
You'll need to enable "Allow .noinit segment to be placed in external memory"
https://docs.espressif.com/projects/esp ... nal-memory
Re: Finding a method for storing a large data before run time
Posted: Tue Aug 19, 2025 7:27 am
by leanh1234
Sorry, I don't understand what you wrote.
You create your own partition by
creating your own partition table .csv file. The easiest way to do that is probably to take one of the default partition tables from the documentation and modify it. In that .csv file, you have to add a line with the desired name of the partition, its type (e.g. "data", or your own custom type in the range 0x40-0xFE), possibly a subtype, and the size of the partition.
You then select your "Custom partition table CSV" via menuconfig and build+flash your project.
Thanks for your patient! As you say, I created memory region via partition table, then writting the data in runtime. However, I saw that skainet example (eg: en_speech_commands_recognition (
https://github.com/espressif/esp-skaine ... ecognition)) always have 2 times writing to flash, that means a mount of data was written to a region in partition created before instead of runtime.
Re: Finding a method for storing a large data before run time
Posted: Tue Aug 19, 2025 9:41 am
by MicroController
You can use
esptool.py to write any data to any location in flash, including to your custom partition. You just have to tell it the address of your partition in flash, i.e. the "offset" from the partition table.
Re: Finding a method for storing a large data before run time
Posted: Thu Aug 21, 2025 4:39 am
by leanh1234
You can use
esptool.py to write any data to any location in flash, including to your custom partition. You just have to tell it the address of your partition in flash, i.e. the "offset" from the partition table.
Thanks for your response. Your advice is useful to me. I totally resolve my problem
Re: Finding a method for storing a large data before run time
Posted: Thu Aug 21, 2025 9:06 am
by vvb333007
Code: Select all
// Call this at startup
void copy_to_psram()
[/quote]
..or add __attribute__((constructor)).
Flash partition can be mmaped as well.