Finding a method for storing a large data before run time

[leanh1234]

Hi guys!

I am trying to find a method to store a large hex array in flash before run time. I also want to move that data to PSRAM in ESP32S3. Do you have any solution for storing the hex array in flash first?
Thank you very much. I hope that I will get any advice from you guys!

Best regards,
Ant Leo

[vvb333007]

You have to flash your ESP32S3 with a firmware, which creates required huge hex files on the flash using fopen, fwrite and fclose. Then you can flash your real firmware which will load that data from flash memory to the PSRAM.

Also take look at mmap functions of ESP-IDF: they allow for memory mapping of a flash partition , so you can access your flash partition as an array, without need to move your data to the PSRAM

[MicroController]

You can create a partition for your data which can be flashed independently of your application. Using the partitions API, your application can then copy the data from the partition in flash to PSRAM.

Alternatively, you can embed binary data in the application.

[leanh1234]

Thanks for your suggestion! It is very informative to me. I also wonder to write the data to a segment in partition in the flash time because I saw that almost skainet examples have 2 times, one is for the main program, the other is for the model such as multinet, vad, afe. If it is imposible, how do I modify my project? I still hope get the advice from you guys! Much Love!

[MicroController]

Sorry, I don't understand what you wrote.

You create your own partition by creating your own partition table .csv file. The easiest way to do that is probably to take one of the default partition tables from the documentation and modify it. In that .csv file, you have to add a line with the desired name of the partition, its type (e.g. "data", or your own custom type in the range 0x40-0xFE), possibly a subtype, and the size of the partition.
You then select your "Custom partition table CSV" via menuconfig and build+flash your project.

[thefury]

Easiest thing to do is probably store the data array in flash .rodata (with the keyword const), and have a data array in PSRAM (EXT_RAM_NOINIT_ATTR) and copy it yourself at startup. There is no way to have it transparently store the data in flash and then copy to PSRAM at startup, although that would be neato.

Code: Select all

#include "esp_attr.h"
#define BIG_DATA_BUFFER_SIZE (69420)
static const uint8_t large_data[BIG_DATA_BUFFER_SIZE] = {
// your data here
};
static EXT_RAM_NOINIT_ATTR uint8_t large_data_in_psram[BIG_DATA_BUFFER_SIZE];

// Call this at startup
void copy_to_psram()
{
    memcpy(large_data_in_psram, large_data, sizeof(large_data));
}

You'll need to enable "Allow .noinit segment to be placed in external memory"

https://docs.espressif.com/projects/esp ... nal-memory

[leanh1234]

Sorry, I don't understand what you wrote.

You create your own partition by creating your own partition table .csv file. The easiest way to do that is probably to take one of the default partition tables from the documentation and modify it. In that .csv file, you have to add a line with the desired name of the partition, its type (e.g. "data", or your own custom type in the range 0x40-0xFE), possibly a subtype, and the size of the partition.
You then select your "Custom partition table CSV" via menuconfig and build+flash your project.

Thanks for your patient! As you say, I created memory region via partition table, then writting the data in runtime. However, I saw that skainet example (eg: en_speech_commands_recognition (https://github.com/espressif/esp-skaine ... ecognition)) always have 2 times writing to flash, that means a mount of data was written to a region in partition created before instead of runtime.

[MicroController]

You can use esptool.py to write any data to any location in flash, including to your custom partition. You just have to tell it the address of your partition in flash, i.e. the "offset" from the partition table.

[leanh1234]

You can use esptool.py to write any data to any location in flash, including to your custom partition. You just have to tell it the address of your partition in flash, i.e. the "offset" from the partition table.

Thanks for your response. Your advice is useful to me. I totally resolve my problem

[vvb333007]

Code: Select all

// Call this at startup
void copy_to_psram()
[/quote]

..or add __attribute__((constructor)).

Flash partition can be mmaped as well.