Skip to content

Commit 2ccf78a

Browse files
projectgusdpgeorge
projectgus
authored and
dpgeorge
committed
ci,esp32: Build oldest & newest ESP-IDF versions in CI.
Intended to catch problems where new features don't build in old ESP-IDF. Includes major refactor to the GitHub Actions Workflow for esp32 port, including making a reusable workflow for both Code Size and ESP32 build jobs. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <angus@redyak.com.au>
1 parent 8f24c86 commit 2ccf78a

5 files changed

Lines changed: 95 additions & 58 deletions

File tree

‎.github/actions/setup_esp32/action.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: Setup ESP-IDF for CI
2+
description: Install ESP-IDF
3+
inputs:
4+
idf_ver:
5+
required: true
6+
type: string
7+
ccache_key:
8+
required: true
9+
type: string
10+
11+
runs:
12+
using: "composite"
13+
14+
steps:
15+
- id: python_ver
16+
name: Read the Python version
17+
run: echo PYTHON_VER=py$(python --version | cut -d' ' -f2) | tee "${GITHUB_OUTPUT}"
18+
shell: bash
19+
20+
- name: Cached ESP-IDF install
21+
id: cache_esp_idf
22+
uses: actions/cache@v5
23+
with:
24+
path: |
25+
./esp-idf/
26+
~/.espressif/
27+
!~/.espressif/dist/
28+
~/.cache/pip/
29+
# Cache is keyed on both IDF version (from the job) and Python version (from the runner)
30+
key: esp-idf-${{ inputs.idf_ver }}-${{ steps.python_ver.outputs.PYTHON_VER }}
31+
32+
- name: Install ESP-IDF packages
33+
if: steps.cache_esp_idf.outputs.cache-hit != 'true'
34+
env:
35+
IDF_VER: ${{ inputs.idf_ver }}
36+
run: tools/ci.sh esp32_idf_setup
37+
shell: bash
38+
39+
- name: ccache
40+
uses: hendrikmuhs/ccache-action@v1.2
41+
with:
42+
key: esp32-${{ inputs.idf_ver }}-${{ inputs.ccache_key }}
43+
44+
- name: Enable CCache for ESP-IDF
45+
run: echo "IDF_CCACHE_ENABLE=1" >> ${GITHUB_ENV}
46+
shell: bash
47+

‎.github/workflows/code_size.yml‎

Lines changed: 9 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -32,28 +32,17 @@ jobs:
3232
- name: Install packages
3333
run: tools/ci.sh code_size_setup
3434

35-
# Needs to be kept in synch with ports_esp32.yml
36-
- id: idf_ver
37-
name: Read the ESP-IDF version (including Python version) and set outputs.IDF_VER
38-
run: tools/ci.sh esp32_idf_ver | tee "${GITHUB_OUTPUT}"
39-
- name: Cached ESP-IDF install
40-
id: cache_esp_idf
41-
uses: actions/cache@v5
42-
with:
43-
path: |
44-
./esp-idf/
45-
~/.espressif/
46-
!~/.espressif/dist/
47-
~/.cache/pip/
48-
key: esp-idf-${{ steps.idf_ver.outputs.IDF_VER }}
49-
- name: Install ESP-IDF packages
50-
if: steps.cache_esp_idf.outputs.cache-hit != 'true'
51-
run: tools/ci.sh esp32_idf_setup
35+
- name: Find IDF_NEWEST_VER
36+
id: idf_ver
37+
run: |
38+
echo "IDF_VER="$(yq .env.IDF_NEWEST_VER < .github/workflows/ports_esp32.yml) \
39+
| tee "${GITHUB_OUTPUT}"
5240
53-
- name: ccache
54-
uses: hendrikmuhs/ccache-action@v1.2
41+
- name: Setup ESP-IDF
42+
uses: ./.github/actions/setup_esp32
5543
with:
56-
key: code_size
44+
idf_ver: ${{ steps.idf_ver.outputs.IDF_VER }}
45+
ccache_key: code_size
5746

5847
- name: Build
5948
run: tools/ci.sh code_size_build

‎.github/workflows/ports_esp32.yml‎

Lines changed: 26 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -17,45 +17,47 @@ concurrency:
1717
group: ${{ github.workflow }}-${{ github.ref }}
1818
cancel-in-progress: true
1919

20+
env:
21+
# Oldest and newest supported ESP-IDF versions, should match ports/esp32/README.md
22+
IDF_OLDEST_VER: &oldest "v5.3"
23+
IDF_NEWEST_VER: &newest "v5.5.1"
24+
2025
jobs:
2126
build_idf:
2227
strategy:
2328
fail-fast: false
2429
matrix:
30+
idf_ver:
31+
- *oldest
32+
- *newest
2533
ci_func: # names are functions in ci.sh
2634
- esp32_build_cmod_spiram_s2
2735
- esp32_build_s3_c3
2836
- esp32_build_c2_c5_c6
2937
- esp32_build_p4
38+
exclude:
39+
# Exclude some jobs on the oldest IDF version, to save resources
40+
- idf_ver: *oldest
41+
ci_func: esp32_build_c2_c5_c6
42+
- idf_ver: *oldest
43+
ci_func: esp32_build_p4
3044
runs-on: ubuntu-latest
3145
steps:
3246
- uses: actions/checkout@v6
3347

34-
# Needs to be kept in synch with code_size.yml
35-
- id: idf_ver
36-
name: Read the ESP-IDF version (including Python version) and set outputs.IDF_VER
37-
run: tools/ci.sh esp32_idf_ver | tee "${GITHUB_OUTPUT}"
48+
# Only the newest IDF version will build the ESP-IDF lockfiles correctly,
49+
# so we need to disable MICROPY_MAINTAINER_BUILD on older versions.
50+
- name: Disable extra checks for older ESP-IDF
51+
id: check_newest_ver
52+
if: ${{ matrix.idf_ver != env.IDF_NEWEST_VER }}
53+
run: echo "MICROPY_MAINTAINER_BUILD=0" >> ${GITHUB_ENV}
3854

39-
- name: Cached ESP-IDF install
40-
id: cache_esp_idf
41-
uses: actions/cache@v5
42-
with:
43-
path: |
44-
./esp-idf/
45-
~/.espressif/
46-
!~/.espressif/dist/
47-
~/.cache/pip/
48-
key: esp-idf-${{ steps.idf_ver.outputs.IDF_VER }}
49-
50-
- name: Install ESP-IDF packages
51-
if: steps.cache_esp_idf.outputs.cache-hit != 'true'
52-
run: tools/ci.sh esp32_idf_setup
53-
54-
# Needs to be kept in synch with code_size.yml
55-
- name: ccache
56-
uses: hendrikmuhs/ccache-action@v1.2
55+
- name: Setup ESP-IDF
56+
uses: ./.github/actions/setup_esp32
5757
with:
58-
key: esp32-${{ matrix.ci_func }}
58+
idf_ver: ${{ matrix.idf_ver }}
59+
ccache_key: ${{ matrix.ci_func }}
60+
5961

60-
- name: Build ci_${{matrix.ci_func }}
62+
- name: Build ci_${{matrix.ci_func }} on ESP-IDF ${{ matrix.idf_ver }}
6163
run: tools/ci.sh ${{ matrix.ci_func }}

‎ports/esp32/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,12 @@ The ESP-IDF changes quickly and MicroPython only supports certain versions. The
5555
current recommended version of ESP-IDF for MicroPython is v5.5.1. MicroPython
5656
also supports v5.3, v5.4, v5.4.1 and v5.4.2.
5757

58+
<!-- Important: If updating the above, please also update:
59+
* IDF_OLDEST_VER & IDF_NEWEST_VER in .github/workflows/port_esp32.yml
60+
* minimum version at bottom of main/idf_component.yml
61+
* the lockfiles/ subdirectory.
62+
-->
63+
5864
To install the ESP-IDF the full instructions can be found at the
5965
[Espressif Getting Started guide](https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/index.html#installation-step-by-step).
6066

‎tools/ci.sh‎

Lines changed: 7 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,9 @@ fi
1010
ulimit -n 1024
1111

1212
# Fail on some things which are warnings otherwise
13-
export MICROPY_MAINTAINER_BUILD=1
13+
if [ -z "$MICROPY_MAINTAINER_BUILD" ]; then
14+
export MICROPY_MAINTAINER_BUILD=1
15+
fi
1416

1517
########################################################################################
1618
# general helper functions
@@ -206,20 +208,11 @@ function ci_embedding_build {
206208
########################################################################################
207209
# ports/esp32
208210

209-
# GitHub tag of ESP-IDF to use for CI, extracted from the esp32 dependency lockfile
210-
# This should end up as a tag name like vX.Y.Z
211-
# (note: This hacky parsing can be replaced with 'yq' once Ubuntu >=24.04 is in use)
212-
IDF_VER=v$(grep -A10 "idf:" ports/esp32/lockfiles/dependencies.lock.esp32 | grep "version:" | head -n1 | sed -E 's/ +version: //')
213-
PYTHON=$(command -v python3 2> /dev/null)
214-
PYTHON_VER=$(${PYTHON:-python} --version | cut -d' ' -f2)
215-
216-
export IDF_CCACHE_ENABLE=1
217-
218-
function ci_esp32_idf_ver {
219-
echo "IDF_VER=${IDF_VER}-py${PYTHON_VER}"
220-
}
221-
222211
function ci_esp32_idf_setup {
212+
if [ -z "$IDF_VER" ]; then
213+
echo "IDF_VER environment variable must be set before running"
214+
return 1
215+
fi
223216
echo "Using ESP-IDF version $IDF_VER"
224217
git clone --depth 1 --branch $IDF_VER https://github.com/espressif/esp-idf.git
225218
# doing a treeless clone isn't quite as good as --shallow-submodules, but it

0 commit comments

Comments
 (0)