NVS split one partition into sub-partitions using nvs_flash_init_partition_ptr()

[leschge]

Our current partition table is already in field and can not be changed anymore. We have one NVS partition but now need three NVS partitions. Using namespaces is not bearable as either of these partition should be erasable and flashable as binary file.

My idea (which seems to work) is the following: Having one defined NVS partition in the partition table and split it into three smaller partitions using nvs_flash_init_partition_ptr() with "virtual" partitions, each at another offset. I know this is not a clean or good approach, but am I missing any underlying implementation that may cause a problem/corrupted NVS using this method?
Of course, these "virtual partitions" must not overlap and must be 4KB aligned.


Here is my working code:

Code: Select all

#include <stdio.h>
#include <inttypes.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_system.h"
#include "esp_log.h"
#include "nvs_flash.h"
#include "nvs.h"

static const char *TAG = "nvs_example";


#define N_VPARTITIONS 3
#define PARTITION_OFFSET 0x50000 #Address of pyhsical NVS partition
#define VPARTITION_SIZE 0x10000 #Virtual sub partition size

// Structure for virtual NV1
esp_partition_t config1_virtual_part = {
    .address = PARTITION_OFFSET,
    .size = VPARTITION_SIZE,
    .label = "nvs_1", // The unique name you will use in NVS calls
    .type = ESP_PARTITION_TYPE_DATA,
    .subtype = ESP_PARTITION_SUBTYPE_DATA_NVS,
    .flash_chip = NULL // Important: Inherit the flash chip reference
};

// Structure for virtual NV2
esp_partition_t config2_virtual_part = {
    .address = PARTITION_OFFSET + VPARTITION_SIZE,
    .size = VPARTITION_SIZE,
    .label = "nvs_2", // The unique name you will use in NVS calls
    .type = ESP_PARTITION_TYPE_DATA,
    .subtype = ESP_PARTITION_SUBTYPE_DATA_NVS,
    .flash_chip = NULL
};

// Structure for virtual NV3
esp_partition_t config3_virtual_part = {
    .address = PARTITION_OFFSET + VPARTITION_SIZE * 2,
    .size = VPARTITION_SIZE,
    .label = "nvs_3", // The unique name you will use in NVS calls
    .type = ESP_PARTITION_TYPE_DATA,
    .subtype = ESP_PARTITION_SUBTYPE_DATA_NVS,
    .flash_chip = NULL
};

typedef struct{
  esp_partition_t* partition;
  nvs_handle_t handle;
  int32_t start_counter;
  const char* key;
} nvs_entry_t;

nvs_entry_t virtual_nvs[N_VPARTITIONS] = {
  {&config1_virtual_part, NULL, 100, "counter_1" },
  {&config2_virtual_part, NULL, 200, "counter_2" },
  {&config3_virtual_part, NULL, 300, "counter_3" }
} ;

void app_main(void)
{
    esp_err_t ret;
    int32_t read_counter = 0;
    
    // Init + Open
    for(uint8_t i = 0; i < N_VPARTITIONS; i++)
    {
        ret = nvs_flash_init_partition_ptr(virtual_nvs[i].partition);
        ESP_LOGI(TAG, "nvs_flash_init_partition (#%d): %s", i, esp_err_to_name(ret));
        
        ret = nvs_open_from_partition(virtual_nvs[i].partition->label , "namespace", NVS_READWRITE, &virtual_nvs[i].handle);
        ESP_LOGI(TAG, "nvs_open (#%d): %s", i, esp_err_to_name(ret));
    }

    // Read + Write
    for(uint8_t i = 0; i < N_VPARTITIONS; i++)
    {
          ESP_LOGI(TAG, "Reading counter from NVS (#%d)...", i);
          ret = nvs_get_i32(virtual_nvs[i].handle, virtual_nvs[i].key, &read_counter);
          switch (ret) {
              case ESP_OK:
                  ESP_LOGI(TAG, "Read counter (#%d) = %li", i, read_counter);
                  break;
              case ESP_ERR_NVS_NOT_FOUND:
                  ESP_LOGW(TAG, "The value is not initialized yet for (#%d)!", i);
                  read_counter = virtual_nvs[i].start_counter;
                  break;
              default:
                  ESP_LOGE(TAG, "Error (%s) reading from NVS (#%d)!", esp_err_to_name(ret), i);
          }
          
          ret = nvs_set_i32(virtual_nvs[i].handle, virtual_nvs[i].key, read_counter+1);
          if (ret != ESP_OK) { ESP_LOGE(TAG, "Failed to write counter for (#%d)!", i); }
          else { ESP_LOGI(TAG, "Write counter to NVS (#%d)...", i); }
      
          ESP_LOGI(TAG, "Committing updates in NVS (#%d)...", i);
          ret = nvs_commit(virtual_nvs[i].handle);
          if (ret != ESP_OK) { ESP_LOGE(TAG, "Failed to commit NVS (#%d) changes!", i); }
      

    }

    // Close
    for(uint8_t i = 0; i < N_VPARTITIONS; i++)
    {
          nvs_close(virtual_nvs[i].handle);
          ESP_LOGI(TAG, "NVS (#%d) handle closed.", i);
    }
}

This is the log:

I (313) main_task: Calling app_main()
I (333) nvs_example: nvs_flash_init_partition (#0): ESP_OK
I (333) nvs_example: nvs_open (#0): ESP_OK
I (353) nvs_example: nvs_flash_init_partition (#1): ESP_OK
I (353) nvs_example: nvs_open (#1): ESP_OK
I (363) nvs_example: nvs_flash_init_partition (#2): ESP_OK
I (363) nvs_example: nvs_open (#2): ESP_OK
I (363) nvs_example: Reading counter from NVS (#0)...
I (373) nvs_example: Read counter (#0) = 101
I (393) nvs_example: Write counter to NVS (#0)...
I (393) nvs_example: Committing updates in NVS (#0)...
I (393) nvs_example: Reading counter from NVS (#1)...
I (393) nvs_example: Read counter (#1) = 201
I (403) nvs_example: Write counter to NVS (#1)...
I (403) nvs_example: Committing updates in NVS (#1)...
I (413) nvs_example: Reading counter from NVS (#2)...
I (413) nvs_example: Read counter (#2) = 301
I (433) nvs_example: Write counter to NVS (#2)...
I (433) nvs_example: Committing updates in NVS (#2)...
I (433) nvs_example: NVS (#0) handle closed.
I (443) nvs_example: NVS (#1) handle closed.
I (443) nvs_example: NVS (#2) handle closed.
I (443) main_task: Returned from app_main()

Also reading the flash seems to look good:

virutal NVS partitions.jpg (246.8 KiB) Viewed 1178 times

[ESP_rrtandler]

@leschge
This hack will work for now. Please be aware of following pitfalls:

• If you are planning to use wifi, it internally, by default, uses the partition labelled "nvs". It means it will initialise it correctly via esp_partition API. This will interfere with your solution if the partition you are "virtually" further parting was originally labelled "nvs". You may encounter the data being reinitialised / overwritten or other inconsistencies. You can let wifi doesn't use NVS to store calibration and similar data at all using menuconfig option WIFI_NVS_ENABLED.
• During the future development of NVS, especially the underlying storage media stack, the function

  Code: Select all

  nvs_flash_init_partition_ptr()

  may become deprecated.

[leschge]

@ESP_rrtandler thanks a lot, this is some great advice.