Does use of newlib nano format reduce stack usage?

jsam589
Posts: 74
Joined: Sat Aug 17, 2019 9:31 pm

Does use of newlib nano format reduce stack usage?

Postby jsam589 » Sun May 31, 2020 5:17 pm

I am looking for ways to reduce the amount of stack needed for tasks. It seems that even a simple task that does logging needs 3 KByte minimum of stack. I can get by without printing 64-bit integers. However, I must have logging. Will changing my config to use newlib nano format reduce the stack used for vprintf, printf, etc.?

Are there other general techniques to reduce stack usage?

Background:
I have about 12 application tasks and also use WiFi, NimBLE and esp-azure. Even though I use external SPIRAM for as much data as possible, I am at the limit for internal RAM.

Thanks!

mikemoy
Posts: 599
Joined: Fri Jan 12, 2018 9:10 pm

Re: Does use of newlib nano format reduce stack usage?

Postby mikemoy » Mon Jun 01, 2020 1:32 am

Not to be rude, but in the time it took you to write the post to ask the question, you could have already re-compiled it using nano to see for yourself.

jsam589
Posts: 74
Joined: Sat Aug 17, 2019 9:31 pm

Re: Does use of newlib nano format reduce stack usage?

Postby jsam589 » Mon Jun 01, 2020 2:23 am

LOL - I posted the question first, knowing it usually takes a while to get an answer. The engineers especially usually are very helpful, but sometimes takes a day or two to hear back. So, in parallel, I then did the test, which also required me to eliminate several %lld and %llx printf's in my code. Funny thing, once I built it and ran it, I found that mbed-tls seems to choke on nano and got several errors during runtime. So, it seems nano may not be an option anyway - I don't want plan to dig into mbed-tls.

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Does use of newlib nano format reduce stack usage?

Postby ESP_Angus » Tue Jun 02, 2020 4:10 am

Hi jsam,

The short answer is yes - you should see a reduction in stack usage from logging if using newlib nano format. I don't have a number at hand, the exact answer will depend on exactly which formatting function(s) are being called with what format arguments.

Newlib nano formatting is a supported configuration so if you're seeing failures in mbedTLS caused by enabling it then please let us know the details of what is happening, as it may be a bug.

Angus

jsam589
Posts: 74
Joined: Sat Aug 17, 2019 9:31 pm

Re: Does use of newlib nano format reduce stack usage?

Postby jsam589 » Thu Jun 04, 2020 1:49 am

Well, since you have asked... I am seeing a consistent error with mbed-tls for esp-azure connection to cloud. My code has been working well before I tried compiling with newlib nano. IDF is v4.0 release. I do not have in any of my code %lld or %llx in any printf-type functions.

A trick I learned on this forum a few months ago was to enable CONFIG_MBEDTLS_DEBUG so I did that and below is the output. The error occurs as the handshaking begins.

Code: Select all

2020-06-04T04:38:26+03:00, D, 27785, "esp-tls", "host:Carrion-IOTHub.azure-devices.net: strlen 34"
2020-06-04T04:38:26+03:00, D, 27982, "esp-tls", "connecting..."
2020-06-04T04:38:27+03:00, D, 28521, "esp-tls", "handshake in progress..."
2020-06-04T04:38:27+03:00, I, 28523, "mbedtls", "ssl_tls.c:8084 => handshake
"
2020-06-04T04:38:27+03:00, I, 28526, "mbedtls", "ssl_cli.c:3510 client state: 0
"
2020-06-04T04:38:27+03:00, I, 28535, "mbedtls", "ssl_tls.c:2755 => flush output
"
2020-06-04T04:38:27+03:00, I, 28543, "mbedtls", "ssl_tls.c:2767 <= flush output
"
2020-06-04T04:38:27+03:00, I, 28551, "mbedtls", "ssl_cli.c:3510 client state: 1
"
2020-06-04T04:38:27+03:00, I, 28559, "mbedtls", "ssl_tls.c:2755 => flush output
"
2020-06-04T04:38:27+03:00, I, 28568, "mbedtls", "ssl_tls.c:2767 <= flush output
"
2020-06-04T04:38:27+03:00, I, 28576, "mbedtls", "ssl_cli.c:774 => write client hello
"
2020-06-04T04:38:27+03:00, D, 28585, "mbedtls", "ssl_cli.c:812 client hello, max version: [3:3]
"
2020-06-04T04:38:27+03:00, D, 28594, "mbedtls", "ssl_cli.c:703 client hello, current time: 1591234707
"
2020-06-04T04:38:27+03:00, D, 28603, "mbedtls", "ssl_cli.c:821 dumping 'client hello, random bytes' (32 bytes)
"
2020-06-04T04:38:27+03:00, D, 28614, "mbedtls", "ssl_cli.c:821 0000:  5e d8 50 93 a2 83 92 26 60 93 57 69 00 4c c9 84  ^.P....&`.Wi.L..
"
2020-06-04T04:38:27+03:00, D, 28626, "mbedtls", "ssl_cli.c:821 0010:  a0 41 13 d8 a2 12 9d 47 35 db 0f 49 51 e1 a0 ef  .A.....G5..IQ...
"
2020-06-04T04:38:27+03:00, D, 28639, "mbedtls", "ssl_cli.c:874 client hello, session id len.: 0
"
2020-06-04T04:38:27+03:00, D, 28648, "mbedtls", "ssl_cli.c:875 dumping 'client hello, session id' (0 bytes)
"
2020-06-04T04:38:27+03:00, D, 28658, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02c
"
2020-06-04T04:38:27+03:00, D, 28667, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c030
"
2020-06-04T04:38:27+03:00, D, 28676, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 009f
"
2020-06-04T04:38:27+03:00, D, 28684, "LED DISP", "LED Que cmd 1, patn 3"
2020-06-04T04:38:27+03:00, D, 28687, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0ad
"
2020-06-04T04:38:27+03:00, D, 28702, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c09f
"
2020-06-04T04:38:27+03:00, D, 28711, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c024
"
2020-06-04T04:38:27+03:00, D, 28720, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c028
"
2020-06-04T04:38:27+03:00, D, 28730, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 006b
"
2020-06-04T04:38:27+03:00, D, 28739, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c00a
"
2020-06-04T04:38:27+03:00, D, 28748, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c014
"
2020-06-04T04:38:27+03:00, D, 28757, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 0039
"
2020-06-04T04:38:27+03:00, D, 28767, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0af
"
2020-06-04T04:38:27+03:00, D, 28776, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0a3
"
2020-06-04T04:38:27+03:00, D, 28785, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02b
"
2020-06-04T04:38:27+03:00, D, 28794, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02f
"
2020-06-04T04:38:27+03:00, D, 28804, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 009e
"
2020-06-04T04:38:27+03:00, D, 28813, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0ac
"
2020-06-04T04:38:27+03:00, D, 28822, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c09e
"
2020-06-04T04:38:27+03:00, D, 28832, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c023
"
2020-06-04T04:38:27+03:00, D, 28841, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c027
"
2020-06-04T04:38:27+03:00, D, 28850, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 0067
"
2020-06-04T04:38:27+03:00, D, 28859, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c009
"
2020-06-04T04:38:27+03:00, D, 28869, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c013
"
2020-06-04T04:38:27+03:00, D, 28878, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 0033
"
2020-06-04T04:38:27+03:00, D, 28887, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0ae
"
2020-06-04T04:38:27+03:00, D, 28897, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0a2
"
2020-06-04T04:38:27+03:00, D, 28906, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 009d
"
2020-06-04T04:38:27+03:00, D, 28915, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c09d
"
2020-06-04T04:38:27+03:00, D, 28924, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 003d
"
2020-06-04T04:38:27+03:00, D, 28934, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 0035
"
2020-06-04T04:38:27+03:00, D, 28943, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c032
"
2020-06-04T04:38:27+03:00, D, 28952, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02a
"
2020-06-04T04:38:27+03:00, D, 28962, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c00f
"
2020-06-04T04:38:27+03:00, D, 28971, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02e
"
2020-06-04T04:38:27+03:00, D, 28980, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c026
"
2020-06-04T04:38:27+03:00, D, 28989, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c005
"
2020-06-04T04:38:27+03:00, D, 28999, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0a1
"
2020-06-04T04:38:27+03:00, D, 29008, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 009c
"
2020-06-04T04:38:27+03:00, D, 29017, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c09c
"
2020-06-04T04:38:27+03:00, D, 29027, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 003c
"
2020-06-04T04:38:27+03:00, D, 29036, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: 002f
"
2020-06-04T04:38:27+03:00, D, 29045, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c031
"
2020-06-04T04:38:27+03:00, D, 29054, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c029
"
2020-06-04T04:38:27+03:00, D, 29064, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c00e
"
2020-06-04T04:38:27+03:00, D, 29073, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c02d
"
2020-06-04T04:38:28+03:00, D, 29082, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c025
"
2020-06-04T04:38:28+03:00, D, 29092, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c004
"
2020-06-04T04:38:28+03:00, D, 29101, "mbedtls", "ssl_cli.c:922 client hello, add ciphersuite: c0a0
"
2020-06-04T04:38:28+03:00, D, 29110, "mbedtls", "ssl_cli.c:934 client hello, got 48 ciphersuites (excluding SCSVs)
"
2020-06-04T04:38:28+03:00, D, 29121, "mbedtls", "ssl_cli.c:943 adding EMPTY_RENEGOTIATION_INFO_SCSV
"
2020-06-04T04:38:28+03:00, D, 29130, "mbedtls", "ssl_cli.c:992 client hello, compress len.: 1
"
2020-06-04T04:38:28+03:00, D, 29139, "mbedtls", "ssl_cli.c:994 client hello, compress alg.: 0
"
2020-06-04T04:38:28+03:00, D, 29148, "mbedtls", "ssl_cli.c:69 client hello, adding server name extension: Carrion-IOTHub.azure-devices.net
"
2020-06-04T04:38:28+03:00, D, 29161, "mbedtls", "ssl_cli.c:186 client hello, adding signature_algorithms extension
"
2020-06-04T04:38:28+03:00, D, 29172, "mbedtls", "ssl_cli.c:271 client hello, adding supported_elliptic_curves extension
"
2020-06-04T04:38:28+03:00, D, 29183, "mbedtls", "ssl_cli.c:336 client hello, adding supported_point_formats extension
"
2020-06-04T04:38:28+03:00, D, 29194, "mbedtls", "ssl_cli.c:518 client hello, adding encrypt_then_mac extension
"
2020-06-04T04:38:28+03:00, D, 29204, "mbedtls", "ssl_cli.c:552 client hello, adding extended_master_secret extension
"
2020-06-04T04:38:28+03:00, D, 29215, "mbedtls", "ssl_cli.c:585 client hello, adding session ticket extension
"
2020-06-04T04:38:28+03:00, D, 29225, "mbedtls", "ssl_cli.c:1071 client hello, total extension length: 115
"
2020-06-04T04:38:28+03:00, I, 29235, "mbedtls", "ssl_tls.c:3184 => write handshake message
"
2020-06-04T04:38:28+03:00, I, 29245, "mbedtls", "ssl_tls.c:3343 => write record
"
2020-06-04T04:38:28+03:00, D, 29252, "mbedtls", "ssl_tls.c:3423 output record: msgtype = 22, version = [3:1], msglen = 258
"
2020-06-04T04:38:28+03:00, I, 29265, "mbedtls", "ssl_tls.c:2755 => flush output
"
2020-06-04T04:38:28+03:00, I, 29272, "mbedtls", "ssl_tls.c:2774 message length: 263, out_left: 263
"
2020-06-04T04:38:28+03:00, I, 29284, "mbedtls", "ssl_tls.c:2779 ssl->f_send() returned 263 (-0xfffffef9)
"
2020-06-04T04:38:28+03:00, I, 29292, "mbedtls", "ssl_tls.c:2807 <= flush output
"
2020-06-04T04:38:28+03:00, I, 29300, "mbedtls", "ssl_tls.c:3476 <= write record
"
2020-06-04T04:38:28+03:00, I, 29309, "mbedtls", "ssl_tls.c:3320 <= write handshake message
"
2020-06-04T04:38:28+03:00, I, 29318, "mbedtls", "ssl_cli.c:1106 <= write client hello
"
2020-06-04T04:38:28+03:00, I, 29327, "mbedtls", "ssl_cli.c:3510 client state: 2
"
2020-06-04T04:38:28+03:00, I, 29335, "mbedtls", "ssl_tls.c:2755 => flush output
"
2020-06-04T04:38:28+03:00, I, 29343, "mbedtls", "ssl_tls.c:2767 <= flush output
"
2020-06-04T04:38:28+03:00, I, 29351, "mbedtls", "ssl_cli.c:1499 => parse server hello
"
2020-06-04T04:38:28+03:00, I, 29360, "mbedtls", "ssl_tls.c:4311 => read record
"
2020-06-04T04:38:28+03:00, I, 29368, "mbedtls", "ssl_tls.c:2536 => fetch input
"
2020-06-04T04:38:28+03:00, I, 29377, "mbedtls", "ssl_tls.c:2697 in_left: 0, nb_want: 5
"
2020-06-04T04:38:28+03:00, I, 29386, "mbedtls", "ssl_tls.c:2721 in_left: 0, nb_want: 5
"
2020-06-04T04:38:28+03:00, I, 29394, "mbedtls", "ssl_tls.c:8094 <= handshake
"
2020-06-04T04:38:28+03:00, D, 29502, "esp-tls", "handshake in progress..."
2020-06-04T04:38:28+03:00, I, 29505, "mbedtls", "ssl_tls.c:8084 => handshake
"
2020-06-04T04:38:28+03:00, I, 29508, "mbedtls", "ssl_cli.c:3510 client state: 2
"
2020-06-04T04:38:28+03:00, I, 29515, "mbedtls", "ssl_tls.c:2755 => flush output
"
2020-06-04T04:38:28+03:00, I, 29524, "mbedtls", "ssl_tls.c:2767 <= flush output
"
2020-06-04T04:38:28+03:00, I, 29532, "mbedtls", "ssl_cli.c:1499 => parse server hello
"
2020-06-04T04:38:28+03:00, I, 29541, "mbedtls", "ssl_tls.c:4311 => read record
"
2020-06-04T04:38:28+03:00, I, 29549, "mbedtls", "ssl_tls.c:2536 => fetch input
"
2020-06-04T04:38:28+03:00, I, 29557, "mbedtls", "ssl_tls.c:2697 in_left: 0, nb_want: 5
"
2020-06-04T04:38:28+03:00, I, 29567, "mbedtls", "ssl_tls.c:2721 in_left: 0, nb_want: 5
"
2020-06-04T04:38:28+03:00, I, 29575, "mbedtls", "ssl_tls.c:2722 ssl->f_recv(_timeout)() returned 0 (-0x0000)
"
2020-06-04T04:38:28+03:00, W, 29585, "mbedtls", "ssl_tls.c:4973 mbedtls_ssl_fetch_input() returned -29312 (-0x7280)
"
2020-06-04T04:38:28+03:00, W, 29597, "mbedtls", "ssl_tls.c:4344 ssl_get_next_record() returned -29312 (-0x7280)
"
2020-06-04T04:38:28+03:00, W, 29608, "mbedtls", "ssl_cli.c:1506 mbedtls_ssl_read_record() returned -29312 (-0x7280)
"
2020-06-04T04:38:28+03:00, I, 29619, "mbedtls", "ssl_tls.c:8094 <= handshake
"
2020-06-04T04:38:28+03:00, E, 29627, "esp-tls", "mbedtls_ssl_handshake returned -0x7280"
2020-06-04T04:38:29+03:00, D, 30683, "LED DISP", "LED Que cmd 1, patn 1"

jsam589
Posts: 74
Joined: Sat Aug 17, 2019 9:31 pm

Re: Does use of newlib nano format reduce stack usage?

Postby jsam589 » Sun Jun 07, 2020 1:36 am

Additional: I installed v4.1-beta2 IDF and tried to use nano and still getting same result. It will build but when azure starts running, it has the same tls error at the same point in code.

ESP_Mahavir
Posts: 188
Joined: Wed Jan 24, 2018 6:51 am

Re: Does use of newlib nano format reduce stack usage?

Postby ESP_Mahavir » Tue Jun 09, 2020 6:45 am

Hi Jsam,

I was not able to reproduce this issue at my end.

- Example used was one from https://github.com/espressif/esp-azure/ ... ample_mqtt
- SDK Configuration had `CONFIG_NEWLIB_NANO_FORMAT=y`
- Tried with both ESP-IDF latest master and v4.1-Beta2 release

Could you please confirm that you can reproduce this issue with above mentioned (pristine) example? Also it would be great if you can provide your `sdkconfig` file.

Additional note:
- Newlib nano configuration can help to reduce stack usage by ~600-800 bytes based on usage pattern.

Mahavir

Who is online

Users browsing this forum: johboh, Majestic-12 [Bot] and 107 guests