ESP-IDF used as Submodule like ESP-ADF

mikemoy
Posts: 599
Joined: Fri Jan 12, 2018 9:10 pm

ESP-IDF used as Submodule like ESP-ADF

Postby mikemoy » Tue Jan 14, 2020 3:40 am

In another thread @chegewara had what I thought was a great idea. But I am not sure how to implement it.
He said:
Lets look at esp-adf project for example. Esp-idf is used as submodule with particular commit:
https://github.com/espressif/esp-adf

I believe you can build any project you want the same way and instead of having global esp-idf, you can have esp-idf per project.
I like this idea. The reason being that looking in the future I can see a potential problem arising where projects that are built today and released using say v3.3.1. But later ESP-IDF will be on another version say v4.x.x. Should my customer want to make changes a year later I need to build that project built on v3.3.1 but my current setup is on v4.x.x which wont work.

Did all that makes sense ;-) How can I copy v3.3.1 into the each project so I know its all self contained much like what ESP-ADF does

chegewara
Posts: 2207
Joined: Wed Jun 14, 2017 9:00 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby chegewara » Tue Jan 14, 2020 8:40 am

1) create project and git init
2) ad esp-idf as submodule https://stackoverflow.com/questions/420 ... repository
3) go to esp-idf and git commit to version you want
4) git add . & git push

https://github.com/chegewara/commit-test

Remember to configure CMakeLists.txt to use esp-idf from subfolder.

mikemoy
Posts: 599
Joined: Fri Jan 12, 2018 9:10 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby mikemoy » Tue Jan 14, 2020 3:26 pm

Thank you for the example. But this did not go smooth sailing for me on the PC.
My default Windows IDF_PATH is C:\Users\Mike\ESP32\esp-idf

Its not clear to me in your example how its pointing to the new submodule.
The CMakeLists.txt file contains:

Code: Select all

# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(hello-world)
The way i read this is that it would be still using the defaul path and not pointing to the included ESP-IDF folder.
Am i just not seeing the light here or what ?

mrrosen
Posts: 18
Joined: Tue Sep 26, 2017 12:53 am

Re: ESP-IDF used as Submodule like ESP-ADF

Postby mrrosen » Tue Jan 14, 2020 6:51 pm

For one of my projects, I did this exact thing however, this was right during the shift to cmake so I did it in the old Makefile system; like so (where I created the esp-idf as a submodule in the idf directory):

Code: Select all

export IDF_PATH = $(PWD)/idf

PROJECT_NAME := my_project

include $(IDF_PATH)/make/project.mk
Pretty sure you could do that same thing with cmake by setting the IDF_PATH environment variable at the start of the file:

Code: Select all

cmake_minimum_required(VERSION 3.5)

set(ENV{IDF_PATH} ${CMAKE_CURRENT_LIST_DIR}/idf)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(my_project)
Last edited by mrrosen on Wed Jan 15, 2020 6:05 pm, edited 1 time in total.

chegewara
Posts: 2207
Joined: Wed Jun 14, 2017 9:00 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby chegewara » Wed Jan 15, 2020 12:14 am

https://github.com/espressif/esp-adf/bl ... .txt#L6-L8

I didnt try it yet, but i believe you need to set IDF_PATH without if, just this line:

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)

mikemoy
Posts: 599
Joined: Fri Jan 12, 2018 9:10 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby mikemoy » Wed Jan 15, 2020 4:24 am

Thanks guys for the help. Didn't work for me. I got much closer with

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)
At the moment I just don't have the time to dig into this.

chegewara
Posts: 2207
Joined: Wed Jun 14, 2017 9:00 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby chegewara » Fri Jan 17, 2020 11:56 am

Ok, ive been overthinking it, but it is very simple. I am using PIO and from PIO terminal i had to do this:

Code: Select all

. ./esp-idf/export.sh
idf.py build
It was with esp-idf v3.3 as submodule described earlier.

PS no changes needed in CMakeLists.txt i mentioned earlier, just forget it

phatpaul
Posts: 109
Joined: Fri Aug 24, 2018 1:14 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby phatpaul » Wed Aug 10, 2022 8:51 pm

I had been doing this since IDF 3 with make. I had a custom build script.
When I ported to IDF 4 with CMake, I created a new init scripts to set the environment.
(My CMakeLists.txt doesn't need anything special for this b/c my init script sets IDF_PATH in the environment. It still uses include("$ENV{IDF_PATH}/tools/cmake/project.cmake"))

I made a windows version and a linux of the script. It expects the esp-idf to be a git submodule in the project repo/tools/esp-idf/.
You can just double-click the _START_IDF_ENV.cmd file to start a cmd window.
On linux you have to type: source sourceme_IDF_ENV.sh
  1. @REM Script for launching ESP-IDF environment on Windows
  2. @REM Author: Paul Abbott, 2022
  3. @ECHO OFF
  4. TITLE ESP-IDF
  5. SETLOCAL EnableDelayedExpansion
  6.  
  7. : %CD% gives current dir
  8. SET THISDIR=%CD%
  9. : %~dp0 gives the directory of this batch file (with a trailing \)
  10. SET SCRIPTDIR=%~dp0
  11.  
  12. :: if using IDF installed globally ::
  13. : SET "IDF_INIT_SCRIPT=C:\Espressif\idf_cmd_init.bat"
  14. :: See C:\Espressif\esp_idf.json for installed IDF version strings
  15. :: Version 4.4
  16. : SET "IDF_VERS_STRING=esp-idf-8de2bd0d9cffd2eca3d3f8442939a034"
  17. :: Version 4.3
  18. : SET "IDF_VERS_STRING=esp-idf-7a3cc3545977e8fdecfa73521e1f3180"
  19.  
  20. :: if using IDF inside repo as submodule::
  21. SET "IDF_INIT_SCRIPT=%SCRIPTDIR%..\tools\esp-idf\install.bat"
  22. SET "IDF_INIT_SCRIPT2=%SCRIPTDIR%..\tools\esp-idf\export.bat"
  23.  
  24. IF NOT EXIST %IDF_INIT_SCRIPT% (
  25.     MORE %SCRIPTDIR%readme.md
  26.     ECHO Script %IDF_INIT_SCRIPT% does not exist!.
  27.     GOTO :DIE
  28. )
  29. CALL %IDF_INIT_SCRIPT% %IDF_VERS_STRING%
  30. IF !ERRORLEVEL! NEQ 0 goto :DIE
  31. IF NOT EXIST %IDF_INIT_SCRIPT% GOTO :DONE
  32.  
  33. CALL %IDF_INIT_SCRIPT2%
  34. IF !ERRORLEVEL! NEQ 0 goto :DIE
  35.  
  36. :DONE
  37. GOTO :SHELL
  38.  
  39. :DIE
  40. : Color the screen red
  41. COLOR 4f
  42. ECHO ERROR!
  43. PAUSE
  44. GOTO :QUIT
  45.  
  46. :QUIT
  47. : Restore the Color of the screen
  48. COLOR 07
  49. GOTO :eof
  50.  
  51. :SHELL
  52. : Restore the Color of the screen
  53. COLOR 07
  54. ECHO/
  55. ECHO   idf build flash monitor -p COM15
  56. ECHO/
  57. cmd /k
  1. #!/bin/bash
  2.  
  3. # Script for setting up ESP_IDF environment on Ubuntu
  4. # use it like this: `source sourceme_IDF_ENV.sh`
  5. # Author: Paul Abbott, 2022
  6. SCRIPT_VER="1.0"
  7. SCRIPT_DIR=$(realpath "$(dirname "${BASH_SOURCE[0]}")")
  8. PWD=$(pwd)
  9.  
  10. # this only works when sourced (not run as a script), because it is adding variables into the local shell
  11. [[ "${BASH_SOURCE[0]}" == "${0}" ]] && echo "Error, must source this script!  Try: \"source ${BASH_SOURCE[0]}\"" && exit 1
  12.  
  13. source $SCRIPT_DIR/../tools/esp-idf/export.sh
  14. alias idf="idf.py"
  15.  
  16. # These commands will not execute, but will be in the history for convenience.
  17. history -s idf build
  18. history -s idf build flash monitor -p /dev/ttyUSB0
  19.  

phatpaul
Posts: 109
Joined: Fri Aug 24, 2018 1:14 pm

Re: ESP-IDF used as Submodule like ESP-ADF

Postby phatpaul » Wed Aug 10, 2022 9:03 pm

The scripts I posted above work OK, as long as you have the correct version of esp-idf checked-out as git submodule working copy. But I find it is common that a dev forgot to do git submodule update --init --recursive after pulling other changes that expect a different esp-idf.

So I added another script that checks that all submodules are at their expected versions. This one is called by CMake.
(I adapted this code from https://github.com/iree-org/iree/pull/6568)
  1. cmake_minimum_required(VERSION 3.5)
  2.  
  3. #-------------------------------------------------------------------------------
  4. # Check if git submodules have been initialized.
  5. # This will only run if python3 is available.
  6. #-------------------------------------------------------------------------------
  7. find_package(Python3 COMPONENTS Interpreter QUIET)
  8. if(Python3_FOUND)
  9.   message("check_submodules")
  10.   execute_process(
  11.     COMMAND ${Python3_EXECUTABLE} ../tools/check_submodules.py
  12.     WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
  13.     RESULT_VARIABLE ret
  14.   )
  15.   if(NOT ret EQUAL "0")
  16.     message(FATAL_ERROR "check_submodule_init.py failed, see the logs above")
  17.   endif()
  18. endif()
  19.  
  20. include("$ENV{IDF_PATH}/tools/cmake/project.cmake")
  21. project(poco-fw-ota)
  1. # Copyright 2021 The IREE Authors
  2. #
  3. # Licensed under the Apache License v2.0 with LLVM Exceptions.
  4. # See https://llvm.org/LICENSE.txt for license information.
  5. # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
  6.  
  7. import os
  8. import subprocess
  9. import sys
  10.  
  11.  
  12. def run():
  13.   # No-op if we're not in a git repository.
  14.   try:
  15.     subprocess.check_call(['git', 'rev-parse', '--is-inside-work-tree'],
  16.                           stdout=subprocess.DEVNULL,
  17.                           stderr=subprocess.DEVNULL)
  18.   except:
  19.     return
  20.  
  21.   output = os.popen("git submodule status --recursive")
  22.   submodules = output.readlines()
  23.  
  24.   for submodule in submodules:
  25.     if (submodule.strip()[0] == "-"):
  26.       print(
  27.           "Error: The git submodule '%s' is not initialized. Please run `git submodule update --init --recursive`"
  28.           % (submodule.split()[1]))
  29.       sys.exit(1)
  30.     if (submodule.strip()[0] == "+"):
  31.       print(
  32.           "Warning: The git submodule '%s' is at a different version than expected. Please run `git submodule update --init --recursive`"
  33.           % (submodule.split()[1]))
  34.  
  35. if __name__ == "__main__":
  36.   run()
BTW:
I'm curious if the new idf-component-manager is a better tool for this though. It looks like it can set the required IDF version in your manifest file.
https://docs.espressif.com/projects/esp ... nager.html

Who is online

Users browsing this forum: No registered users and 146 guests