Ways to make binary smaller?

0xffff
Posts: 41
Joined: Tue Jun 19, 2018 1:53 am

Ways to make binary smaller?

Postby 0xffff » Fri Jul 27, 2018 3:25 pm

We're nearing our flash partition size limit with our current binary. From make size-components top 3 libraries are:

Code: Select all

            Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
            libmbedtls.a         88    268      0     131077    43796  175229
           libnet80211.a        314   9055   3601     106478    12066  131514
               liblwip.a         19   3770      0      88423    16421  108633
but early this year it was:

Code: Select all

            Archive File DRAM .data & .bss   IRAM Flash code & rodata   Total
            libmbedtls.a         88    268      0     126361    41772  168489
               liblwip.a         19   3772      0      88355    16466  108612
           libnet80211.a       1852   8275     26      67620     6657   84430
In particular, the 802.11 library has grown by 40K. Are there strategies for making the executable smaller? Have already removed BLE and other components not currently using.

thanks

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: Ways to make binary smaller?

Postby kolban » Sat Jul 28, 2018 8:18 pm

What is your map for partitions? I wanted to make sure that you know that the sizes for executables default to 1MB but can be expanded as desired. It will be a function of whether or not you are using OTA and whether or not you have need of other partitions for application data.
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

0xffff
Posts: 41
Joined: Tue Jun 19, 2018 1:53 am

Re: Ways to make binary smaller?

Postby 0xffff » Sat Jul 28, 2018 8:44 pm

Hi, thanks for response. I do have a custom partition table in which I use 2 partitions of 1MB each for OTA and another partition of 1.5MB for storage. I can make the storage partition smaller and increase the size of the OTA partitions if my executable gets too big, but since we are often doing OTA updates overseas on shaky connections, the larger the app, the greater the chance that the OTA will fail.

I guess I'm just wary of size creep, and was wondering if I there were any tricks besides sdkconfig customization for making the app smaller.

Thanks

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: Ways to make binary smaller?

Postby kolban » Mon Jul 30, 2018 4:21 am

A very experimental technique that i've been tinkering with recently seems to work but I am hesitant to recommend it ... however for completeness, I'll describe it.

If we think about a user application (i.e. an app that you write), it is composed of two primary parts. The first being the ESP-IDF (all the libraries that are supplied by Espressif). The second is your own user app code (the code that you write). If we take the bold assumption that the ESP-IDF is "relatively" static then an OTA push could be just your own user app. If we loosely say that for any given app, 60-90% of the size is ESP-IDF, then the remainder (your own app) is much smaller. What I have found is that we can compile the ESP-IDF stand-alone and produce an app which is "just" ESP-IDF plus a small loader. We can then download and save just our user app into its own partition. We then end up with (for example), one ESP-IDF partition that might be 1MBytes and our user app ends up at 200-300K per copy. So instead of having to need 2x1.3MBytes for OTA (2.6MB) ... I end up with 1x1MB + 2x0.3MB = 1.6MB thereby having shaved off 1MB of total space needed for OTA. The major benefit (to me) was not saving the flash space but rather an OTA new app transmission changed from 1.3MB to 0.3MB and took less than 1/3 of the time/risk.

The obvious downside of this is that (absent an ability to refresh the ESP-IDF) you are stuck with a constant ESP-IDF inside your device.
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Ways to make binary smaller?

Postby ESP_Angus » Mon Jul 30, 2018 4:41 am

Hi 0xfff,

There are a few settings you can change to reduce overall binary size. libnet80211 is part of the WiFi libraries so its size is not changed by these, but the other components should get a bit smaller:
All of these have some tradeoffs for functionality, which are documented at the links above. I recommend applying them one at a time, probably in the order given above, and see what helps for your project.

The other (probably obvious) thing to note is that the app will only contain code which is linked into it, ie functions/symbols which are referenced at compile time. The less functionality is compiled in, the smaller the final binary size.

hemant.chaudhari
Posts: 16
Joined: Wed Jun 13, 2018 11:48 am

Re: Ways to make binary smaller?

Postby hemant.chaudhari » Wed Aug 08, 2018 6:01 am

Hello,

You can also enable "compressed upload" in "serial flasher config" in "menuconfig" utility.

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: Ways to make binary smaller?

Postby ESP_Angus » Wed Aug 08, 2018 7:18 am

Hi hemant,
hemant.chaudhari wrote: You can also enable "compressed upload" in "serial flasher config" in "menuconfig" utility.
Unfortunately "compressed upload" only compresses the data "over the wire" for a faster serial transfer. It takes up the same amount of bytes in the flash regardless of whether this option is set.

hemant.chaudhari
Posts: 16
Joined: Wed Jun 13, 2018 11:48 am

Re: Ways to make binary smaller?

Postby hemant.chaudhari » Wed Aug 08, 2018 12:51 pm

Thanks ESP_Angus....

User avatar
fly135
Posts: 606
Joined: Wed Jan 03, 2018 8:33 pm
Location: Orlando, FL

Re: Ways to make binary smaller?

Postby fly135 » Wed Aug 08, 2018 5:47 pm

I also found that HTTPS OTAs are much more likely to fail than HTTP (i.e encrypted vs unencrypted).

John A

User avatar
fly135
Posts: 606
Joined: Wed Jan 03, 2018 8:33 pm
Location: Orlando, FL

Re: Ways to make binary smaller?

Postby fly135 » Wed Aug 08, 2018 5:52 pm

kolban wrote:A very experimental technique that i've been tinkering with recently seems to work but I am hesitant to recommend it ... however for completeness, I'll describe it.
How does the app without the support libraries link to the app with the support libraries? This sounds really interesting and useful. Like having DLLs. But I don't see how the user app "links" to the minimal app IDF code.

John A

Who is online

Users browsing this forum: MicroController and 103 guests