[Solved] Certificate loaded from NVS causes an mbedTLS error

[dmitrij999]

Hello!

I'm using NVS for certificates providing to HTTPS server, and NVS files are generated by mfg_gen.py script.
The certificate files are OK, but after adding it into NVS file with further loading on ESP32, I see certificate and some bloat symbols.
Additionally, I see the cert length on ESP32 loaded from NVS different with original cert length. The same situation with key.
Could you please help to solve this issue?

P.S. Embedding certificate into firmware directly is not eligible for me, it's needed for mass manufacture files generation

Certificate file in attachments

Logs and codes:

Certificate is loaded, but some bloat symbols and errors when client tries to connect to ESP32 HTTPS server:

Code: Select all

I (00:00:02.767) HTTPS Server: Starting server
I (00:00:02.794) STORAGE: Loaded cert: 
-----BEGIN CERTIFICATE-----
MIIFPzCCAyegAwIBAgICEBwwDQYJKoZIhvcNAQELBQAwcTELMAkGA1UEBhMCUlUx
DzANBgNVBAgMBlJ1c3NpYTETMBEGA1UECgwKR3JvbGxpIEx0ZDE8MDoGA1UEAwwz
R3JvbGxpIEx0ZCBJbnRlcm1lZGlhdGUgQ0EgZm9yIGRldmljZSBIVFRQUyBzZXJ2
ZXJzMB4XDTIyMDUxNDE3MDUzNloXDTMyMDUxMTE3MDUzNlowgZAxCzAJBgNVBAYT
AlJVMQ8wDQYDVQQIDAZSdXNzaWExDzANBgNVBAcMBlJ1c3NpYTEsMCoGA1UECgwj
R3JvbGxpIEx0ZCBIVFRQUyBzZXJ2ZXIgY2VydGlmaWNhdGUxEzARBgNVBAsMCkdy
b2xsaSBMdGQxHDAaBgNVBAMME2dyb2xsaS1hYmMwMTMubG9jYWwwgZ8wDQYJKoZI
hvcNAQEBBQADgY0AMIGJAoGBAMB9CDP1GDbnx0LvWA0wCLPqX/u2zExqGv+3F6Sw
aGGaMwVKhMONBHonkKF6J03ASobqpsn4gNDZ1hEftZP0qpJuwre0nSK/pC8RhwHQ
u2glGsKUeefpZKB/aaWPsHwDS/qdge+0qTl25uy65TcK7VM62peBwz2XbtaV7Kd6
GgUHAgMBAAGjggFDMIIBPzAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIGQDAz
BglghkgBhvhCAQ0EJhYkT3BlblNTTCBHZW5lcmF0ZWQgU2VydmVyIENlcnRpZmlj
YXRlMB0GA1UdDgQWBBSqW1r5nH4VYHfhh6lPPRKrKT59djCBpQYDVR0jBIGdMIGa
gBTYKy5/MTDdRdxXMxZu40hgZw+Pe6F+pHwwejELMAkGA1UEBhMCUlUxDzANBgNV
BAgMBlJ1c3NpYTEPMA0GA1UEBwwGUnVzc2lhMRMwEQYDVQQKDApHcm9sbGkgTHRk
MTQwMgYDVQQDDCtHcm9sbGkgTHRkIFJvb3QgQ0EgZm9yIGRldmljZSBIVFRQUyBz
ZXJ2ZXJzggIQATAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwEw
DQYJKoZIhvcNAQELBQADggIBAAHUXMwwQLOHp/SUmd0zwWOzFnVYzYJupMMZuV2c
fH7JbVXAZGBnmNdIboy0VoY7NJtco7j1fj5/nzn/xhAsMBFFDVUh1VA4e8LtLOWZ
g7oJriBQOSaofgwYRJL8QR5SeIxNiEbGq8KDKqAnAA4+Ss3uj8xzRKRgHf4OG6di
xI8MUmkIXKxuIYo+EeAEVO1R4Ep9cWUXetFWnyTeipOiD95IK6OU9Xp0cKGEtbLC
nWpA7GvIEmtrXudRdJ/HP8tCgim50SxmG1bCztIPdGd65p/hhqdsvrApVaElbpGM
iyFAyA600WGTUKpekdmnHmCunS14yk6vuP5FGojt+M98cpUZjQAdp9P9D9hDanpC
LiJuz4+mtpUJ7UY3tVBrfGEefT+1qhPHOcK2oxm4xSOQNgdk8/mggV6ttcl0ACln
Fsnrv3iTsaHPbDF8a3oDS9q7Yk1a4OXmngTXoS8rnggeu0wMm7WHRUB2k7Nr0W0H
z9duzZq0QngovuPL2wSzSz6+haawtvvlgYiuzNy/+D8V84WUKyu7GBuOyVWTMxtn
HmkMxURLJx/ONppxKPyHeRm4uYNVDzf5Ry/EHrG+Eq8t9nsfWK2spIYOZli2Azga
wyO/MD4OKoBOyC3RfbTEJWRbHry6rriX3tU3hbRG3kuydJzSkRlaKhyev9yIyy2c
qAdO
-----END CERTIFICATE-----
?
Length: 1880

-------------------------------

I (00:00:51.396) esp_https_server: performing session handshake
E (00:00:51.399) esp-tls-mbedtls: mbedtls_x509_crt_parse returned -0x2180
E (00:00:51.400) esp-tls-mbedtls: Failed to set server pki context
E (00:00:51.407) esp-tls-mbedtls: Failed to set server configurations, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (00:00:51.419) esp-tls-mbedtls: create_ssl_handle failed, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (00:00:51.430) esp_https_server: esp_tls_create_server_session failed
W (00:00:51.440) httpd: httpd_accept_conn: session creation failed
W (00:00:51.445) httpd: httpd_server: error accepting new connection

Certificate length


Part of mfg_config.csv

Code: Select all

certs_ns,namespace,
https_cert,file,binary
https_key,file,binary

Reading certs from NVS:

Code: [Select all] [Expand/Collapse]

1. typedef struct {
2.     unsigned char https_server_cert[3072];
3.     uint32_t https_server_cert_len;
4.     unsigned char https_server_key[3072];
5.     uint32_t https_server_key_len;
6.     unsigned char ota_root_cert[2048];
7.     uint32_t ota_root_cert_len;
8. } device_certs;
9.  
10. unsigned char * getBlobNVSValue(nvs_handle_t handler, const char* key, unsigned char * default_value) {
11.     unsigned char * out = default_value;
12.  
13.     size_t required_size;
14.     esp_err_t err;
15.  
16.     err = nvs_get_blob(handler, key, NULL, &required_size);
17.     if (err) return out;
18.  
19.     unsigned char value[required_size];
20.     err = nvs_get_blob(handler, key, value, &required_size);
21.     if (err) return out;
22.     out = value;
23.  
24.     return out;
25. }
26.  
27. device_certs readDeviceCertsFromNVS() {
28.     device_certs out;
29.     out.https_server_cert_len = 0;
30.     out.https_server_key_len = 0;
31.     out.ota_root_cert_len = 0;
32.  
33.     flush_string((char *)out.https_server_cert, 3072);
34.     flush_string((char *)out.https_server_key, 3072);
35.  
36.     nvs_handle_t my_handle;
37.     esp_err_t err = nvs_open(CERTS_NAMESPACE, NVS_READWRITE, &my_handle);
38.     if (err != ESP_OK) {
39.         ESP_LOGE(STORAGE_TAG, "Error (%s) opening NVS handle to obtain certs!", esp_err_to_name(err));
40.     } else {
41.         //printf("Done\n");
42.         ESP_LOGD(STORAGE_TAG, "Done opening storage for %s, reading", CERTS_NAMESPACE);
43.  
44.         strcpy((char *)out.https_server_cert, (const char *)getBlobNVSValue(my_handle, HTTPS_SERVER_CERT, (unsigned char *)""));
45.         strcpy((char *)out.https_server_key, (const char *)getBlobNVSValue(my_handle, HTTPS_SERVER_KEY, (unsigned char *)""));
46.         //strcpy((char *)out.https_server_cert, getStringNVSValue(my_handle, HTTPS_SERVER_CERT, (const char *)""));
47.         //strcpy((char *)out.https_server_key, getStringNVSValue(my_handle, HTTPS_SERVER_KEY, (const char *)""));
48.         strcpy((char *)out.ota_root_cert, (const char *)getBlobNVSValue(my_handle, OTA_ROOT_CERT, (unsigned char *)""));
49.         out.https_server_cert_len = strlen((const char *)out.https_server_cert);
50.         out.https_server_key_len = strlen((const char *)out.https_server_key);
51.         out.ota_root_cert_len = strlen((const char *)out.ota_root_cert);
52.        
53.         // Close
54.         nvs_close(my_handle);
55.     }
56.     ESP_LOGD(STORAGE_TAG, "Got certs lengths: HTTPS server cert %u key %u OTA root cert %u", out.https_server_cert_len, out.https_server_key_len, out.ota_root_cert_len);
57.     ESP_LOGI(STORAGE_TAG, "Loaded cert: \n%s\nLength: %d", out.https_server_cert, out.https_server_cert_len);
58.     ESP_LOGI(STORAGE_TAG, "Loaded key: \n%s\nLength: %d", out.https_server_key, out.https_server_key_len);
59.     return out;
60. }

GeSHi © Codebox Plus Extension

[dmitrij999]

Valid reading is reached with strncpy

Code: [Select all] [Expand/Collapse]

1. unsigned char * getBlobNVSValue(nvs_handle_t handler, const char* key, unsigned char * default_value, size_t &size_of_blob) {
2.     unsigned char * out = default_value;
3.  
4.     size_t required_size = 0;
5.     esp_err_t err;
6.  
7.     err = nvs_get_blob(handler, key, NULL, &required_size);
8.     if (err) return out;
9.    
10.     ESP_LOGD(STORAGE_TAG, "Size of blob, 1st attempt: %d", required_size);
11.    
12.     unsigned char value[required_size];
13.     err = nvs_get_blob(handler, key, value, &required_size);
14.     if (err) return out;
15.     out = value;
16.  
17.     ESP_LOGD(STORAGE_TAG, "Size of blob, 2nd attempt: %d", required_size);
18.     size_of_blob = required_size;
19.  
20.     return out;
21. }
22.  
23. //////////////////////////////////////
24.  
25.         strncpy((char *)out.https_server_cert, (const char *)getBlobNVSValue(my_handle, HTTPS_SERVER_CERT, (unsigned char *)"", (size_t&)out.https_server_cert_len), out.https_server_cert_len);
26.         strncpy((char *)out.https_server_key, (const char *)getBlobNVSValue(my_handle, HTTPS_SERVER_KEY, (unsigned char *)"", (size_t&)out.https_server_key_len), out.https_server_key_len);

GeSHi © Codebox Plus Extension

But in debug output, certificate is invalid because of cert format

Code: Select all

I (00:00:44.240) esp_https_server: performing session handshake
E (00:00:44.246) esp-tls-mbedtls: mbedtls_x509_crt_parse returned -0x2180
I (00:00:44.254) esp-tls-mbedtls: (FFFFDE80): X509 - The CRT/CRL/CSR format is invalid, e.g. different type expected
I (00:00:44.262) MAIN: Heap size: 99836
E (00:00:44.265) esp-tls-mbedtls: Failed to set server pki context
E (00:00:44.276) esp-tls-mbedtls: Failed to set server configurations, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (00:00:44.288) esp-tls-mbedtls: create_ssl_handle failed, returned [0x8015] (ESP_ERR_MBEDTLS_X509_CRT_PARSE_FAILED)
E (00:00:44.300) esp_https_server: esp_tls_create_server_session failed

[dmitrij999]

Hello colleagues!

I've tried to embed certificate into firmware, and it was OK with HTTPS server, but if I load the same certificate from NVS, it causes error 0x2180, considering that if I check the cacert_pem value and it was OK.

Have no clue

[ESP_Mahavir]

Hello,

`mbedtls_x509_crt_parse` expects certificate buffer to be null terminated and length including null termination character in case of PEM format. Reference code at https://github.com/espressif/esp-jumpst ... #L311-L326 might help in this context. Can you please check this once?

Thanks.

[dmitrij999]

Solved this issue.

First, I clean buffer with filling with null terminates and read certs with my implementation:

Code: Select all

flush_string((char*)crts.https_server_cert, 3072);
flush_string((char*)crts.https_server_key, 3072);
crts = readDeviceCertsFromNVS();

Then I specify buffer length with cert length+1

Code: Select all

conf.cacert_pem = crts.https_server_cert;
conf.cacert_len = (size_t)crts.https_server_cert_len+1;
conf.prvtkey_pem = crts.https_server_key;
conf.prvtkey_len = (size_t)crts.https_server_key_len+1;

Thanks!