ESP-IDF used as Submodule like ESP-ADF

[mikemoy]

In another thread @chegewara had what I thought was a great idea. But I am not sure how to implement it.
He said:

Lets look at esp-adf project for example. Esp-idf is used as submodule with particular commit:
https://github.com/espressif/esp-adf

I believe you can build any project you want the same way and instead of having global esp-idf, you can have esp-idf per project.

I like this idea. The reason being that looking in the future I can see a potential problem arising where projects that are built today and released using say v3.3.1. But later ESP-IDF will be on another version say v4.x.x. Should my customer want to make changes a year later I need to build that project built on v3.3.1 but my current setup is on v4.x.x which wont work.

Did all that makes sense How can I copy v3.3.1 into the each project so I know its all self contained much like what ESP-ADF does

[chegewara]

1) create project and git init
2) ad esp-idf as submodule https://stackoverflow.com/questions/420 ... repository
3) go to esp-idf and git commit to version you want
4) git add . & git push

https://github.com/chegewara/commit-test

Remember to configure CMakeLists.txt to use esp-idf from subfolder.

[mikemoy]

Thank you for the example. But this did not go smooth sailing for me on the PC.
My default Windows IDF_PATH is C:\Users\Mike\ESP32\esp-idf

Its not clear to me in your example how its pointing to the new submodule.
The CMakeLists.txt file contains:

Code: Select all

# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(hello-world)

The way i read this is that it would be still using the defaul path and not pointing to the included ESP-IDF folder.
Am i just not seeing the light here or what ?

[mrrosen]

For one of my projects, I did this exact thing however, this was right during the shift to cmake so I did it in the old Makefile system; like so (where I created the esp-idf as a submodule in the idf directory):

Code: Select all

export IDF_PATH = $(PWD)/idf

PROJECT_NAME := my_project

include $(IDF_PATH)/make/project.mk

Pretty sure you could do that same thing with cmake by setting the IDF_PATH environment variable at the start of the file:

Code: Select all

cmake_minimum_required(VERSION 3.5)

set(ENV{IDF_PATH} ${CMAKE_CURRENT_LIST_DIR}/idf)

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(my_project)

[chegewara]

https://github.com/espressif/esp-adf/bl ... .txt#L6-L8

I didnt try it yet, but i believe you need to set IDF_PATH without if, just this line:

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)

[mikemoy]

Thanks guys for the help. Didn't work for me. I got much closer with

Code: Select all

set(ENV{IDF_PATH} "${PROJECT_PATH}/esp-idf/")
set(EXTRA_COMPONENT_DIRS ${PROJECT_PATH}/components)

At the moment I just don't have the time to dig into this.

[chegewara]

Ok, ive been overthinking it, but it is very simple. I am using PIO and from PIO terminal i had to do this:

Code: Select all

. ./esp-idf/export.sh
idf.py build

It was with esp-idf v3.3 as submodule described earlier.

PS no changes needed in CMakeLists.txt i mentioned earlier, just forget it

[phatpaul]

I had been doing this since IDF 3 with make. I had a custom build script.
When I ported to IDF 4 with CMake, I created a new init scripts to set the environment.
(My CMakeLists.txt doesn't need anything special for this b/c my init script sets IDF_PATH in the environment. It still uses include("$ENV{IDF_PATH}/tools/cmake/project.cmake"))

I made a windows version and a linux of the script. It expects the esp-idf to be a git submodule in the project repo/tools/esp-idf/.
You can just double-click the _START_IDF_ENV.cmd file to start a cmd window.
On linux you have to type: source sourceme_IDF_ENV.sh

Code: _START_IDF_ENV.cmd Select all

@REM Script for launching ESP-IDF environment on Windows

@REM Author: Paul Abbott, 2022

@ECHO OFF

TITLE ESP-IDF

SETLOCAL EnableDelayedExpansion



: %CD% gives current dir

SET THISDIR=%CD%

: %~dp0 gives the directory of this batch file (with a trailing \)

SET SCRIPTDIR=%~dp0



:: if using IDF installed globally ::

: SET "IDF_INIT_SCRIPT=C:\Espressif\idf_cmd_init.bat"

:: See C:\Espressif\esp_idf.json for installed IDF version strings

:: Version 4.4

: SET "IDF_VERS_STRING=esp-idf-8de2bd0d9cffd2eca3d3f8442939a034"

:: Version 4.3

: SET "IDF_VERS_STRING=esp-idf-7a3cc3545977e8fdecfa73521e1f3180"



:: if using IDF inside repo as submodule::

SET "IDF_INIT_SCRIPT=%SCRIPTDIR%..\tools\esp-idf\install.bat"

SET "IDF_INIT_SCRIPT2=%SCRIPTDIR%..\tools\esp-idf\export.bat"



IF NOT EXIST %IDF_INIT_SCRIPT% (

    MORE %SCRIPTDIR%readme.md

    ECHO Script %IDF_INIT_SCRIPT% does not exist!.

    GOTO :DIE

)

CALL %IDF_INIT_SCRIPT% %IDF_VERS_STRING%

IF !ERRORLEVEL! NEQ 0 goto :DIE

IF NOT EXIST %IDF_INIT_SCRIPT% GOTO :DONE



CALL %IDF_INIT_SCRIPT2%

IF !ERRORLEVEL! NEQ 0 goto :DIE



:DONE

GOTO :SHELL



:DIE

: Color the screen red

COLOR 4f

ECHO ERROR!

PAUSE

GOTO :QUIT



:QUIT

: Restore the Color of the screen

COLOR 07

GOTO :eof



:SHELL

: Restore the Color of the screen

COLOR 07

ECHO/ 

ECHO   idf build flash monitor -p COM15

ECHO/ 

cmd /k

Code: sourceme_IDF_ENV.sh Select all

#!/bin/bash



# Script for setting up ESP_IDF environment on Ubuntu

# use it like this: `source sourceme_IDF_ENV.sh`

# Author: Paul Abbott, 2022

SCRIPT_VER="1.0"

SCRIPT_DIR=$(realpath "$(dirname "${BASH_SOURCE[0]}")")

PWD=$(pwd)



# this only works when sourced (not run as a script), because it is adding variables into the local shell

[[ "${BASH_SOURCE[0]}" == "${0}" ]] && echo "Error, must source this script!  Try: \"source ${BASH_SOURCE[0]}\"" && exit 1



source $SCRIPT_DIR/../tools/esp-idf/export.sh

alias idf="idf.py"



# These commands will not execute, but will be in the history for convenience.

history -s idf build

history -s idf build flash monitor -p /dev/ttyUSB0

[phatpaul]

The scripts I posted above work OK, as long as you have the correct version of esp-idf checked-out as git submodule working copy. But I find it is common that a dev forgot to do git submodule update --init --recursive after pulling other changes that expect a different esp-idf.

So I added another script that checks that all submodules are at their expected versions. This one is called by CMake.
(I adapted this code from https://github.com/iree-org/iree/pull/6568)

Code: CMakeLists.txt Select all

cmake_minimum_required(VERSION 3.5)



#-------------------------------------------------------------------------------

# Check if git submodules have been initialized.

# This will only run if python3 is available.

#-------------------------------------------------------------------------------

find_package(Python3 COMPONENTS Interpreter QUIET)

if(Python3_FOUND)

  message("check_submodules")

  execute_process(

    COMMAND ${Python3_EXECUTABLE} ../tools/check_submodules.py

    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}

    RESULT_VARIABLE ret

  )

  if(NOT ret EQUAL "0")

    message(FATAL_ERROR "check_submodule_init.py failed, see the logs above")

  endif()

endif()



include("$ENV{IDF_PATH}/tools/cmake/project.cmake")

project(poco-fw-ota)

Code: check_submodules.py Select all

# Copyright 2021 The IREE Authors

#

# Licensed under the Apache License v2.0 with LLVM Exceptions.

# See https://llvm.org/LICENSE.txt for license information.

# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception



import os

import subprocess

import sys





def run():

  # No-op if we're not in a git repository.

  try:

    subprocess.check_call(['git', 'rev-parse', '--is-inside-work-tree'],

                          stdout=subprocess.DEVNULL,

                          stderr=subprocess.DEVNULL)

  except:

    return



  output = os.popen("git submodule status --recursive")

  submodules = output.readlines()



  for submodule in submodules:

    if (submodule.strip()[0] == "-"):

      print(

          "Error: The git submodule '%s' is not initialized. Please run `git submodule update --init --recursive`"

          % (submodule.split()[1]))

      sys.exit(1)

    if (submodule.strip()[0] == "+"):

      print(

          "Warning: The git submodule '%s' is at a different version than expected. Please run `git submodule update --init --recursive`"

          % (submodule.split()[1]))



if __name__ == "__main__":

  run()

BTW:
I'm curious if the new idf-component-manager is a better tool for this though. It looks like it can set the required IDF version in your manifest file.
https://docs.espressif.com/projects/esp ... nager.html