ESP Coredump in Flash Cannot be Read

[NevynSelby]

IDF: 5.5
Board: ESP-WROVER-KIT

I have taken the hello_world and added an assert into the code to trigger a failure. The assertion is triggered and the output seems to suggest that coredump is being created.

I then try to read the coredump and the tool set is not reading the coredump from flash.

Application output:

Code: Select all

Hello world!
This is esp32 chip with 2 CPU core(s), WiFi/BTBLE, silicon revision v1.0, 4MB external flash
Minimum free heap size: 297104 bytes
Restarting in 100 seconds...
Restarting in 99 seconds...
Restarting in 98 seconds...
Restarting in 97 seconds...
Restarting in 96 seconds...
Restarting in 95 seconds...
Restarting in 94 seconds...
Restarting in 93 seconds...
Restarting in 92 seconds...
Restarting in 91 seconds...

assert failed: app_main hello_world_main.c:47 (0)


Backtrace: 0x40081709:0x3ffb5650 0x400849b1:0x3ffb5670 0x40089bd9:0x3ffb5690 0x400d4df6:0x3ffb57b0 0x400e27e8:0x3ffb57f0 0x400850b5:0x3ffb5820
--- 0x40081709: panic_abort at /Users/markstevens/esp/esp-idf/components/esp_system/panic.c:469
--- 0x400849b1: esp_system_abort at /Users/markstevens/esp/esp-idf/components/esp_system/port/esp_system_chip.c:87
--- 0x40089bd9: __assert_func at /Users/markstevens/esp/esp-idf/components/newlib/src/assert.c:80
--- 0x400d4df6: app_main at /Users/markstevens/Public/CoreDumpTest/main/hello_world_main.c:47
--- 0x400e27e8: main_task at /Users/markstevens/esp/esp-idf/components/freertos/app_startup.c:208
--- 0x400850b5: vPortTaskWrapper at /Users/markstevens/esp/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:139




ELF file SHA256: a3af34c55f24ebca

I (10240) esp_core_dump_flash: Save core dump to flash...
I (10245) esp_core_dump_common: Backing up stack @ 0x3ffb54d0 and use core dump stack @ 0x3ffb3d30
I (10254) esp_core_dump_flash: Erase flash 8192 bytes @ 0x3b0000
I (10478) esp_core_dump_flash: Write end offset 0x1fc4, check sum length 4
I (10478) esp_core_dump_common: Core dump used 1168 bytes on stack. 3012 bytes left free.
I (10481) esp_core_dump_common: Restoring stack @ 0x3ffb54d0
I (10486) esp_core_dump_flash: Core dump has been saved to flash.
CPU halted.

idf.py coredump-info generates the error: A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfe3 bytes.

Full application output:

Code: Select all

Executing action: coredump-info
Serial port /dev/cu.usbserial-2101
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
WARNING:root:The core dump image offset is not specified. Use partition offset: 0x3b0000.
Failed to load core dump: esptool script execution failed with error 2, failed command was: '['/Users/markstevens/.espressif/python_env/idf5.5_py3.12_env/bin/python', '-m', 'esptool', '-c', 'auto', '-p', '/dev/cu.usbserial-2101', '-b', '460800', 'read_flash', '3866624', '8132', '/var/folders/m4/sw8tgkhs20549s7f__sz62l40000gn/T/tmph5mwpbwq']'

┌────── Additional information about the error: 
│   
│   esptool.py v4.9.0
│   Serial port /dev/cu.usbserial-2101
│   Connecting.........
│   Detecting chip type... Unsupported detection protocol, switching and trying again...
│   Connecting....
│   Detecting chip type... ESP32
│   Chip is ESP32-D0WDQ6 (revision v1.0)
│   Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, BLK3 partially reserved, Coding Scheme 3/4
│   Crystal is 40MHz
│   MAC: 84:0d:8e:18:5b:08
│   Uploading stub...
│   Running stub...
│   Stub running...
│   Changing baud rate to 460800
│   Changed.
│   Configuring flash size...

│   A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfe3 bytes

│   
└────── end of additional information about the error.

Heap section of the configuration file:

Code: Select all

CONFIG_ESP_COREDUMP_ENABLE_TO_FLASH=y
# CONFIG_ESP_COREDUMP_ENABLE_TO_UART is not set
# CONFIG_ESP_COREDUMP_ENABLE_TO_NONE is not set
# CONFIG_ESP_COREDUMP_DATA_FORMAT_BIN is not set
CONFIG_ESP_COREDUMP_DATA_FORMAT_ELF=y
CONFIG_ESP_COREDUMP_CHECKSUM_CRC32=y
# CONFIG_ESP_COREDUMP_CHECKSUM_SHA256 is not set
# CONFIG_ESP_COREDUMP_CAPTURE_DRAM is not set
CONFIG_ESP_COREDUMP_CHECK_BOOT=y
CONFIG_ESP_COREDUMP_ENABLE=y
CONFIG_ESP_COREDUMP_LOGS=y
CONFIG_ESP_COREDUMP_MAX_TASKS_NUM=32
# CONFIG_ESP_COREDUMP_FLASH_NO_OVERWRITE is not set
CONFIG_ESP_COREDUMP_STACK_SIZE=4096
# end of Core dump

Partition file:

Code: Select all

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,      0x9000,   0x6000,
phy_init, data, phy,      0xf000,   0x1000,
factory,  app,  factory,  0x10000,  0x3a0000,
coredump, data, coredump, 0x3b0000, 0x10000,
littlefs, data, spiffs,   0x3c0000, 0x40000,

The long term aim here is to record the coredump and then send this back to a server for later analysis.

Has anyone managed to do anything similar?

Regards,
Mark

[nopnop2002]

There may be restrictions on the offset of the core dump partition.

Try changing the start location of your core dump partition.

Code: Select all

# ESP-IDF Partition Table
# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,      0x9000,   0x6000,
phy_init, data, phy,      0xf000,   0x1000,
coredump, data, coredump,,          0x10000,
factory,  app,  factory,,           0x3a0000,
littlefs, data, spiffs,,            0x40000,

[NevynSelby]

No luck, Command:

Code: Select all

idf.py coredump-info

Output:

Code: Select all

A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0x18d bytes

And these commands:

Code: Select all

parttool.py --port /dev/tty.usbserial-2101 read_partition --partition-name coredump --output coredump.bin

Looks to have worked:

Code: Select all

.
.
Running stub...
Stub running...
Configuring flash size...
65536 (100 %)
65536 (100 %)
Read 65536 bytes at 0x00010000 in 6.0 seconds (87.8 kbit/s)...
Hard resetting via RTS pin...
Read partition 'coredump' contents from device at offset 0x10000 to file 'coredump.bin'

But decoding fails:

Code: Select all

esp-coredump --chip esp32 info_corefile coredump.bin

Results in a python traceback:

Code: Select all

    raise ConstError(f"parsing expected {repr(self.value)} but parsed {repr(obj)}", path=path)
construct.core.ConstError: Error in path (parsing) -> elf_header -> e_ident -> EI_MAG
parsing expected b'\x7fELF' but parsed b'\xc4\x1f\x00\x00'

Regards,
Mark

[nopnop2002]

The following command will analyze the core dump file saved locally.

Code: Select all

$ idf.py coredump-info -c coredump.bin

The idf.py coredump-info and idf.py coredump-debug commands are wrappers around the esp-coredump tool for easier use in the ESP-IDF environment.

I used this:

Code: Select all

$ parttool.py --port /dev/ttyUSB0 read_partition --partition-name coredump --output hoge.bin
.
.
Running stub...
Stub running...
Configuring flash size...
65536 (100 %)
65536 (100 %)
Read 65536 bytes at 0x00110000 in 5.9 seconds (88.7 kbit/s)...
Hard resetting via RTS pin...
Read partition 'coredump' contents from device at offset 0x110000 to file 'hoge.bin'

$ idf.py coredump-info -c hoge.bin

The following commands do almost the same thing.
idf.py coredump-info analyzes the partitions directly.
esp-coredump saves the partitions to a local file and then analyzes that local file.

Code: Select all

$ idf.py coredump-info
Executing action: coredump-info
Serial port /dev/ttyUSB0
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
WARNING:root:The core dump image offset is not specified. Use partition offset: 0x110000.
===============================================================
==================== ESP32 CORE DUMP START ====================



$ esp-coredump --chip esp32 info_corefile build/test.elf
espcoredump.py v1.13.1
INFO: Invoke esptool to read image.
INFO: Retrieving core dump partition offset and size...
INFO: Core dump partition offset=1114112, size=65536

7908 (100 %)
7908 (100 %)
Read 7908 bytes at 0x00110000 in 0.7 seconds (87.9 kbit/s)...
Hard resetting via RTS pin...

===============================================================
==================== ESP32 CORE DUMP START ====================

[NevynSelby]

So the following commands:

Code: Select all

parttool.py --port /dev/tty.usbserial-1101 read_partition --partition-name coredump --output coredump.bin
idf.py coredump-info -c coredump.bin

generates the following output:

Code: Select all

Executing action: coredump-info
===============================================================
==================== ESP32 CORE DUMP START ====================
The ROM ELF file won't load automatically since it was not found for the provided chip type.

So it looks like it is trying to read the file but there is no crash information.

So I have changed the application from using an assert to a NULL reference using *((uint32_t *) 0) = 0;

This time we get register etc as well as the coredump messages in the output from idf.py monitor.

Using the above two commands to retrieve the coredump partition and I get the same output.

Regards,
Mark

[nopnop2002]

I used the following code:

Code: Select all

#include <stdio.h>
#include <inttypes.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"

void app_main()
{
        printf("Now esp is crashed\n");
        assert(0);
}

I used this partition:

Code: Select all

# Name,   Type, SubType, Offset,  Size, Flags
# Note: if you change the phy_init or app partition offset, make sure to change the offset in Kconfig.projbuild
nvs,      data, nvs,     0x9000,  0x6000,
phy_init, data, phy,     0xf000,  0x1000,
factory,  app,  factory,,         1M,
#coredump, data, coredump,,        64K
coredump, data, coredump,,        0x10000,

The Coredump will be saved.

panic-4.jpg (126.09 KiB) Viewed 452 times

Running on esp-idf v5.4.2

Code: Select all

$ idf.py --version
ESP-IDF v5.4.2-dirty

[NevynSelby]

Yes, I see that in running

Code: Select all

idf.py monitor

What I don't see is any information in the retrieved coredump when running these commands

Code: Select all

parttool.py --port /dev/tty.usbserial-1101 read_partition --partition-name coredump --output coredump.bin
idf.py coredump-info -c coredump.bin

[nopnop2002]

Yes, I see that in running

Code: Select all

idf.py monitor

What I don't see is any information in the retrieved coredump when running these commands

Code: Select all

parttool.py --port /dev/tty.usbserial-1101 read_partition --partition-name coredump --output coredump.bin
idf.py coredump-info -c coredump.bin

I think the core dump isn't written to flash correctly.
I want to see the all your logging of idf.py monitor.

This probably won't work correctly.

Code: Select all

CONFIG_ESP_COREDUMP_CHECKSUM_CRC32=y

[NevynSelby]

So changed to SHA256 as you don't trust CRC32.

Full output:

Code: Select all

rst:0x10 (RTCWDT_RTC_RESET),boot:0x3e (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:1396
load:0x40078000,len:13708
load:0x40080400,len:2996
--- 0x40080400: _invalid_pc_placeholder at /Users/markstevens/esp/esp-idf/components/xtensa/xtensa_vectors.S:2235
entry 0x40080574
--- 0x40080574: call_start_cpu0 at /Users/markstevens/esp/esp-idf/components/bootloader/subproject/main/bootloader_start.c:25
I (104) cpu_start: Multicore app
I (113) cpu_start: Pro cpu start user code
I (113) cpu_start: cpu freq: 160000000 Hz
I (113) app_init: Application information:
I (113) app_init: Project name:     hello_world
I (117) app_init: App version:      1
I (120) app_init: Compile time:     Aug  7 2025 14:42:47
I (125) app_init: ELF file SHA256:  a8179f5b9a7fcd8f...
I (130) app_init: ESP-IDF:          v5.5-1-gb66b5448e0-dirty
I (136) efuse_init: Min chip rev:     v0.0
I (140) efuse_init: Max chip rev:     v3.99 
I (144) efuse_init: Chip rev:         v1.0
I (148) heap_init: Initializing. RAM available for dynamic allocation:
I (154) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (159) heap_init: At 3FFB42D8 len 0002BD28 (175 KiB): DRAM
I (164) heap_init: At 3FFE0440 len 00003AE0 (14 KiB): D/IRAM
I (169) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (175) heap_init: At 4008C7EC len 00013814 (78 KiB): IRAM
I (181) spi_flash: detected chip: gd
I (183) spi_flash: flash io: dio
I (187) esp_core_dump_flash: Init core dump to flash
I (192) esp_core_dump_flash: Found partition 'coredump' @ 10000 65536 bytes
I (210) esp_core_dump_flash: Core dump data checksum is correct
I (210) esp_core_dump_flash: Found core dump 7744 bytes in flash @ 0x10000
I (213) main_task: Started on CPU0
I (223) main_task: Calling app_main()
Hello world!
This is esp32 chip with 2 CPU core(s), WiFi/BTBLE, silicon revision v1.0, 4MB external flash
Minimum free heap size: 297072 bytes
Restarting in 100 seconds...
Restarting in 99 seconds...
Restarting in 98 seconds...
Restarting in 97 seconds...
Restarting in 96 seconds...
Restarting in 95 seconds...
Restarting in 94 seconds...
Restarting in 93 seconds...
Restarting in 92 seconds...
Restarting in 91 seconds...
Guru Meditation Error: Core  0 panic'ed (StoreProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x400d4e8d  PS      : 0x00060830  A0      : 0x800e2ba3  A1      : 0x3ffb57d0  
--- 0x400d4e8d: app_main at /Users/markstevens/Public/CoreDumpTest/main/hello_world_main.c:49
A2      : 0x00000003  A3      : 0x3f401cd1  A4      : 0x3f401d19  A5      : 0x000000df  
A6      : 0x3f401cd1  A7      : 0x0000005a  A8      : 0x00000000  A9      : 0x3ffb57b0  
A10     : 0x00000064  A11     : 0x0000005b  A12     : 0x3f403b75  A13     : 0x3f403b4b  
A14     : 0x3f4056a9  A15     : 0x3f4056ae  SAR     : 0x00000004  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xfffffffe  
--- 0x400014fd: strlen in ROM
--- 0x4000150d: strlen in ROM


Backtrace: 0x400d4e8a:0x3ffb57d0 0x400e2ba0:0x3ffb5810 0x4008513d:0x3ffb5840
--- 0x400d4e8a: app_main at /Users/markstevens/Public/CoreDumpTest/main/hello_world_main.c:49
--- 0x400e2ba0: main_task at /Users/markstevens/esp/esp-idf/components/freertos/app_startup.c:208
--- 0x4008513d: vPortTaskWrapper at /Users/markstevens/esp/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:139




ELF file SHA256: a8179f5b9a7fcd8f

I (10289) esp_core_dump_flash: Save core dump to flash...
I (10295) esp_core_dump_common: Backing up stack @ 0x3ffb5650 and use core dump stack @ 0x3ffb3d40
I (10304) esp_core_dump_flash: Erase flash 8192 bytes @ 0x10000
I (10519) esp_core_dump_flash: Write end offset 0x1e40, check sum length 32
I (10520) esp_core_dump_common: Core dump used 1296 bytes on stack. 2884 bytes left free.
I (10523) esp_core_dump_common: Restoring stack @ 0x3ffb5650
I (10528) esp_core_dump_flash: Core dump has been saved to flash.
CPU halted.

And the commands executed:

Code: Select all

parttool.py --port /dev/tty.usbserial-1101 read_partition --partition-name coredump --output coredump.bin
 ✔  17.31 Dur  14:43:39
Running /Users/markstevens/.espressif/python_env/idf5.5_py3.12_env/bin/python /Users/markstevens/esp/esp-idf/components/esptool_py/esptool/esptool.py --port /dev/tty.usbserial-1101 read_flash 32768 3072 /var/folders/m4/sw8tgkhs20549s7f__sz62l40000gn/T/tmpxq7_v3vx...
esptool.py v4.9.0
Serial port /dev/tty.usbserial-1101
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision v1.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, BLK3 partially reserved, Coding Scheme 3/4
Crystal is 40MHz
MAC: 84:0d:8e:18:5b:08
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
3072 (100 %)
3072 (100 %)
Read 3072 bytes at 0x00008000 in 0.3 seconds (80.9 kbit/s)...
Hard resetting via RTS pin...
Running /Users/markstevens/.espressif/python_env/idf5.5_py3.12_env/bin/python /Users/markstevens/esp/esp-idf/components/esptool_py/esptool/esptool.py --port /dev/tty.usbserial-1101 read_flash 65536 65536 coredump.bin...
esptool.py v4.9.0
Serial port /dev/tty.usbserial-1101
Connecting....
Detecting chip type... Unsupported detection protocol, switching and trying again...
Connecting....
Detecting chip type... ESP32
Chip is ESP32-D0WDQ6 (revision v1.0)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, BLK3 partially reserved, Coding Scheme 3/4
Crystal is 40MHz
MAC: 84:0d:8e:18:5b:08
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
65536 (100 %)
65536 (100 %)
Read 65536 bytes at 0x00010000 in 6.0 seconds (87.1 kbit/s)...
Hard resetting via RTS pin...
Read partition 'coredump' contents from device at offset 0x10000 to file 'coredump.bin'

and

Code: Select all

idf.py coredump-info -c coredump.bin
 ✔  11.22 Dur  14:44:08
Executing action: coredump-info
===============================================================
==================== ESP32 CORE DUMP START ====================
The ROM ELF file won't load automatically since it was not found for the provided chip type.

Regards,
Mark

[nopnop2002]

Code: Select all

ELF file SHA256: a8179f5b9a7fcd8f

I (10289) esp_core_dump_flash: Save core dump to flash...
I (10295) esp_core_dump_common: Backing up stack @ 0x3ffb5650 and use core dump stack @ 0x3ffb3d40
I (10304) esp_core_dump_flash: Erase flash 8192 bytes @ 0x10000
I (10519) esp_core_dump_flash: Write end offset 0x1e40, check sum length 32
I (10520) esp_core_dump_common: Core dump used 1296 bytes on stack. 2884 bytes left free.
I (10523) esp_core_dump_common: Restoring stack @ 0x3ffb5650
I (10528) esp_core_dump_flash: Core dump has been saved to flash.
CPU halted.

The core dump appears to be stored correctly in flash.


I don't know the cause of this error.
If you have another ESP32 development board, try it on that board.

Code: Select all

┌────── Additional information about the error: 
│   
│   esptool.py v4.9.0
│   Serial port /dev/cu.usbserial-2101
│   Connecting.........
│   Detecting chip type... Unsupported detection protocol, switching and trying again...
│   Connecting....
│   Detecting chip type... ESP32
│   Chip is ESP32-D0WDQ6 (revision v1.0)
│   Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, BLK3 partially reserved, Coding Scheme 3/4
│   Crystal is 40MHz
│   MAC: 84:0d:8e:18:5b:08
│   Uploading stub...
│   Running stub...
│   Stub running...
│   Changing baud rate to 460800
│   Changed.
│   Configuring flash size...

│   A fatal error occurred: Corrupt data, expected 0x1000 bytes but received 0xfe3 bytes

│   
└────── end of additional information about the error.