Esp delta ota + Secure Boot + Flash Encryption

[renansoares]

Hello everyone!

Recently, I came across the ESP Delta OTA Library and found it very interesting and useful for some projects I work on. Currently, the OTA update process is quite costly because we use mobile networks to download binaries, most of which exceed 1MB. Some projects have the limitation of having only a few megabytes available for internet use.

The problem faced is that these projects use Secure Boot and Flash Encryption. In my tests with Delta OTA, I simply cannot get a signed patch to work with Secure Boot. My question is whether this would be possible?

I have not yet tested with equipment running only Flash Encryption implemented, but I need to first resolve the issue with Secure Boot.

Looking forward to some guidance!

[hmalpani]

Hello
Good to know that you are using ESP Delta OTA component. Can you please describe the workflow you are using to create the patch files when secure boot is also enabled.

[brocliath]

I am also very interested in using the Delta OTA option with Secure Boot and Flash Encryption.

[renansoares]

Sorry for the delay in responding. I did some tests with Delta OTA and realized that it was generating the patch with the wrong binaries. In the end, I managed to make it work in a project with Secure Boot and Flash Encryption.

Some things I did:
In menuconfig, I changed these parameters:

• APP_REPRODUCIBLE_BUILD // ENABLE THIS OPTION
• CONFIG_APP_COMPILE_TIME_DATE // DISABLE THIS OPTION

These changes in menuconfig were to be able to create a patch by reverting the versions;
(To use with Secure Boot): The binaries used to generate the patch must be signed beforehand.

An example update flow to facilitate understanding:

Code: Select all

mkdir -p ota/versions/ ota/patches/

// Considering version 0 as the first BASE
cp release_build/sign/main.bin ota/versions/0.bin

// Modified the code and generated version 1:
cp release_build/sign/main.bin ota/versions/1.bin

// Modified again and generated version 2:
cp release_build/sign/main.bin ota/versions/2.bin

Now I generate a patcher between version 0 (base) and 1:

Code: Select all

cd ota && python esp_delta_ota_patch_gen.py --chip esp32s3 --base_binary versions/0.bin --new_binary versions/1.bin --patch_file_name patches/patch_base_to_new.bin && cd ..
// And I use this patcher (patch_base_to_new.bin) to perform the update;

Now I generate a patcher between version 1 (base) and 2:

Code: Select all

cd ota && python esp_delta_ota_patch_gen.py --chip esp32s3 --base_binary versions/1.bin --new_binary versions/2.bin --patch_file_name patches/patch_base_to_new_1.bin && cd ..
// And I use this patcher (patch_base_to_new_1.bin) to perform the update;

And finally, for testing purposes, I reverted to version 0:

Code: Select all

cd ota && python esp_delta_ota_patch_gen.py --chip esp32s3 --base_binary versions/2.bin --new_binary versions/0.bin --patch_file_name patches/patch_base_to_new_2.bin && cd ..
// And I use this patcher (patch_base_to_new_2.bin) to perform the update, creating the patch between version 2 (base) and 0, making it revert;

[mjgciltd]

Hi everyone and @renansoares

I am having trouble executing the script to generate the patch versions.
I am looking for guidance.

I am using standard toolchain setup on Windows machine and have Desktop Environment with C++ installed with Visual Studio Build Tools.

I tried to use the script, manually installing detools and all the requirements in my system however, the script fails to execute or the installations of packages fail.

1. (Path) C:\Espressif\v5.4.2\idf :-> Cannot see the installations for detools in my system. If I manually install detools I get this:

Code: Select all

"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Espressif\v5.4.2\tools\Espressif\python_env\idf5.4_py3.11_env\include -IC:\Espressif\v5.4.2\tools\Espressif\tools\idf-python\3.11.2\include -IC:\Espressif\v5.4.2\tools\Espressif\tools\idf-python\3.11.2\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" /Tcdetools/libdivsufsort/divsufsort.c /Fobuild\temp.win-amd64-cpython-311\Release\detools\libdivsufsort\divsufsort.obj
      divsufsort.c
      "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Espressif\v5.4.2\tools\Espressif\python_env\idf5.4_py3.11_env\include -IC:\Espressif\v5.4.2\tools\Espressif\tools\idf-python\3.11.2\include -IC:\Espressif\v5.4.2\tools\Espressif\tools\idf-python\3.11.2\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.44.35207\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt" /Tcdetools/sais/sais.c /Fobuild\temp.win-amd64-cpython-311\Release\detools\sais\sais.obj
      sais.c
      detools/sais/sais.c(31): fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.44.35207\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2

2. (Path) Project Folder with the delta_ota_component :-> executing script esp_delta_ota_patch_gen.py generates

Code: Select all

Please install 'detools'. Use command `pip install -r tools/requirements.txt`

Executing pip install -r tools/requirements.txt produces the same error of

Code: Select all

Cannot open include file: 'Python.h'

3. I tried to manually install the detools package using CMD without ESP-IDF env. As it cannot find esptool, it failed. I try installing esptool successfully and retry. This would get:

Code: Select all

Traceback (most recent call last):
  File "C:\dump\esp_delta_ota_patch_gen.py", line 139, in <module>
    main()
  File "C:\dump\esp_delta_ota_patch_gen.py", line 127, in main
    create_patch(args.chip, args.base_binary, args.new_binary, args.patch_file_name)
  File "C:\dump\esp_delta_ota_patch_gen.py", line 57, in create_patch
    x = re.search(r"Validation Hash: ([A-Za-z0-9]+) \(valid\)", content)
                                                                ^^^^^^^
UnboundLocalError: cannot access local variable 'content' where it is not associated with a value

The problem resembles here: https://learn.microsoft.com/en-us/answe ... -include-f

Any help will be appreciated. Thanks.