Get SHA256 of SPIFFS partition?

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Get SHA256 of SPIFFS partition?

Postby dmaxben » Mon Jun 29, 2020 6:44 pm

Im having some trouble getting the ESP to properly calculate the SHA hash of my SPIFFS partition.

Using IDF 3.3.1

This works fine for getting the SHA256 of the running firmware

Code: Select all

uint8_t FWsha_256[HASH_LEN] = {0};
esp_partition_get_sha256(esp_ota_get_running_partition(), FWsha_256);
    calc_sha256(FWsha_256);
    ESP_LOGE(TAG, "spiffs hash: %s", hash_print);
But this returns all zero's for the SPIFFS SHA256

Code: Select all

 const esp_partition_t *_partition = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, NULL);
uint8_t SPIFFSsha_256[HASH_LEN] = {0};
esp_partition_get_sha256(&_partition, SPIFFSsha_256);
    calc_sha256(SPIFFSsha_256);
    ESP_LOGE(TAG, "spiffs hash: %s", hash_print);
=================================================================================

(This is what Im doing to print the hash)

Code: Select all

char hash_print[HASH_LEN * 2 + 1];
#define HASH_LEN 32 /* SHA-256 digest length */

static void calc_sha256(const uint8_t *image_hash)
{
    hash_print[HASH_LEN * 2] = 0;
    for (int i = 0; i < HASH_LEN; ++i)
    {
        sprintf(&hash_print[i * 2], "%02x", image_hash[i]);
    }
}

chegewara
Posts: 2207
Joined: Wed Jun 14, 2017 9:00 pm

Re: Get SHA256 of SPIFFS partition?

Postby chegewara » Mon Jun 29, 2020 7:13 pm

https://github.com/espressif/esp-idf/issues/4586

Fixed in IDF v4.x, not fixed in v3.x.

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Tue Jun 30, 2020 1:19 pm

chegewara wrote:
Mon Jun 29, 2020 7:13 pm
https://github.com/espressif/esp-idf/issues/4586

Fixed in IDF v4.x, not fixed in v3.x.
ahh darn... so theres no way to make it work in IDF 3.x ?

Guess Ill have to work on porting my code to IDF 4.x

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Get SHA256 of SPIFFS partition?

Postby ESP_Angus » Wed Jul 01, 2020 4:44 am

Hi Max,

Thanks for providing a detailed report of what isn't working.

The bug linked on GitHub was an issue introduced in ESP-IDF v4.1 development (regression when we added support for ESP32-S2 Beta), so it shouldn't be a bug in v3.x at all.

I just did a quick test with ESP-IDF v3.3.1 by modifying the SPIFFS example as follows:

Code: Select all

diff --git a/examples/storage/spiffs/main/spiffs_example_main.c b/examples/storage/spiffs/main/spiffs_example_main.c
index 2148139a3f..996d277ca0 100644
--- a/examples/storage/spiffs/main/spiffs_example_main.c
+++ b/examples/storage/spiffs/main/spiffs_example_main.c
@@ -12,6 +12,7 @@
 #include <sys/stat.h>
 #include "esp_err.h"
 #include "esp_log.h"
+#include "esp_partition.h"
 #include "esp_spiffs.h"
 
 static const char *TAG = "example";
@@ -96,4 +97,13 @@ void app_main(void)
     // All done, unmount partition and disable SPIFFS
     esp_vfs_spiffs_unregister(NULL);
     ESP_LOGI(TAG, "SPIFFS unmounted");
+
+    uint8_t hash[32] = {0};
+    const esp_partition_t *p = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, NULL);
+    ESP_ERROR_CHECK( esp_partition_get_sha256(p, hash) );
+    printf("SPIFFS partition hash ");
+    for (int i = 0; i < sizeof(hash); i++) {
+        printf("%02x ", hash[i]);
+    }
+    printf("\n");
 }
and it prints the hash OK:

Code: Select all

SPIFFS partition hash 56 b5 de f3 f4 c9 5d 52 ee b3 e6 0d 24 ef d6 14 8f 72 10 a5 e5 ee 30 77 93 ee a3 0f 99 37 e9 b6 
Confirmed by reading back the whole partition:

Code: Select all

$ esptool.py -b 921600 read_flash 0x110000 0xf0000 spiffs_readback.bin
[...]
$ sha256sum spiffs_readback.bin 
56b5def3f4c95d52eeb3e60d24efd6148f7210a5e5ee307793eea30f9937e9b6  spiffs_readback.bin
I don't immediately see any problem with the code you posted, though.

Would suggest adding some checks: check that _partition pointer is returned with a non-null value, check that _partition->address and _partition->size look correct, and finally check the result of esp_partition_get_sha256 is ESP_OK.

If it turns out this is a bug in ESP-IDF then we will do our best to fix it. v3.3.x is still supported until March 2022 so there is no need to upgrade unless you want to.

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Wed Jul 01, 2020 2:27 pm

ESP_Angus wrote:
Wed Jul 01, 2020 4:44 am
Hi Max,

Thanks for providing a detailed report of what isn't working.

The bug linked on GitHub was an issue introduced in ESP-IDF v4.1 development (regression when we added support for ESP32-S2 Beta), so it shouldn't be a bug in v3.x at all.

I just did a quick test with ESP-IDF v3.3.1 by modifying the SPIFFS example as follows:

Code: Select all

diff --git a/examples/storage/spiffs/main/spiffs_example_main.c b/examples/storage/spiffs/main/spiffs_example_main.c
index 2148139a3f..996d277ca0 100644
--- a/examples/storage/spiffs/main/spiffs_example_main.c
+++ b/examples/storage/spiffs/main/spiffs_example_main.c
@@ -12,6 +12,7 @@
 #include <sys/stat.h>
 #include "esp_err.h"
 #include "esp_log.h"
+#include "esp_partition.h"
 #include "esp_spiffs.h"
 
 static const char *TAG = "example";
@@ -96,4 +97,13 @@ void app_main(void)
     // All done, unmount partition and disable SPIFFS
     esp_vfs_spiffs_unregister(NULL);
     ESP_LOGI(TAG, "SPIFFS unmounted");
+
+    uint8_t hash[32] = {0};
+    const esp_partition_t *p = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, NULL);
+    ESP_ERROR_CHECK( esp_partition_get_sha256(p, hash) );
+    printf("SPIFFS partition hash ");
+    for (int i = 0; i < sizeof(hash); i++) {
+        printf("%02x ", hash[i]);
+    }
+    printf("\n");
 }
and it prints the hash OK:

Code: Select all

SPIFFS partition hash 56 b5 de f3 f4 c9 5d 52 ee b3 e6 0d 24 ef d6 14 8f 72 10 a5 e5 ee 30 77 93 ee a3 0f 99 37 e9 b6 
Confirmed by reading back the whole partition:

Code: Select all

$ esptool.py -b 921600 read_flash 0x110000 0xf0000 spiffs_readback.bin
[...]
$ sha256sum spiffs_readback.bin 
56b5def3f4c95d52eeb3e60d24efd6148f7210a5e5ee307793eea30f9937e9b6  spiffs_readback.bin
I don't immediately see any problem with the code you posted, though.

Would suggest adding some checks: check that _partition pointer is returned with a non-null value, check that _partition->address and _partition->size look correct, and finally check the result of esp_partition_get_sha256 is ESP_OK.

If it turns out this is a bug in ESP-IDF then we will do our best to fix it. v3.3.x is still supported until March 2022 so there is no need to upgrade unless you want to.
Hmmm... does SPIFFS have to be unmounted/unregistered before calculating the SHA256 hash?

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Wed Jul 01, 2020 2:42 pm

Im using this exact code:

Code: Select all

esp_vfs_spiffs_unregister(NULL);
    ESP_LOGI(TAG, "SPIFFS unmounted");

    uint8_t hash[32] = {0};
    const esp_partition_t *p = esp_partition_find_first(ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, NULL);
    ESP_LOGI(TAG, "Found spiffs");
    ESP_LOGI(TAG, "SPIFFS address: %d    spiffs size: %d", p->address, p->size);
    ESP_ERROR_CHECK(esp_partition_get_sha256(p, hash));
    printf("SPIFFS partition hash ");
    for (int i = 0; i < sizeof(hash); i++)
    {
        printf("%02x ", hash[i]);
    }
    printf("\n");

And the ESP32 just crashes...

[ESP32-can.cpp:1714] setup(): SPIFFS unmounted
[ESP32-can.cpp:1718] setup(): Found spiffs
[ESP32-can.cpp:1719] setup(): SPIFFS address: 5308416 spiffs size: 11468800
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Guru Meditation Error: Core 0 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x4008056cGuru Meditation Error: Core 0 panic'ed (LoadStoreError). Exception was unhandled.
Guru Meditation Error: Core 0 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x4008056cGuru Meditation Error: Core 0 panic'ed (LoadStoreError). Exception was unhandled.
Guru Meditation Error: Core 0 panic'ed (IllegalInstruction). Exception was unhandled.
Memory dump at 0x4008056cGuru Meditation Error: Core 0 panic'ed (LoadStoreError). Exception was unhandled.
Guru Meditation Error: Core 0 panic'ed (IllegalInstruction). Exception was unhandled.

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Wed Jul 01, 2020 3:24 pm

Im wondering if Im running into problems simply due to the size of the partition?

Im using a 16MB flash chip, and the SPIFFS partition is 11.4MB.

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Wed Jul 01, 2020 6:06 pm

Ok so after lots of experimentation....

It seems bootloader_mmap (and therefore SHA256 calculation) cant handle chunks >3862528 bytes.

I guess we're out of luck as far as calculating SHA256 hash of large partitions on a 16MB flash chip. :(

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Get SHA256 of SPIFFS partition?

Postby ESP_Angus » Fri Jul 03, 2020 9:26 am

Hi dmaxben,

Yes, this will be it. There is a limit to the total amount of flash which can be mapped at one time in the address space.

This is a bug, we'll fix it (the routine should map the region in chunks to handle this). Not certain if this fix will be backported to v3.3, though.

Angus

dmaxben
Posts: 108
Joined: Thu Nov 16, 2017 6:04 pm

Re: Get SHA256 of SPIFFS partition?

Postby dmaxben » Tue Jul 07, 2020 3:45 pm

ESP_Angus wrote:
Fri Jul 03, 2020 9:26 am
Hi dmaxben,

Yes, this will be it. There is a limit to the total amount of flash which can be mapped at one time in the address space.

This is a bug, we'll fix it (the routine should map the region in chunks to handle this). Not certain if this fix will be backported to v3.3, though.

Angus
Ah ok. Is there any timeline for fixing this bug? Hopefully this will be backported to v3.3, especially since support for 3.3 is guaranteed to 2022...

Who is online

Users browsing this forum: Bing [Bot], Corand and 107 guests