Getting random Interrupt Watchdog errors

[kluverp]

Hi all,

I have an ESP32 running on a custom build PCB. However sometimes the program crashes with the following error message:

Code: Select all

Interrupt wd$
 timeout on CPU0 PC      0021  : 0x0003A0      088e  : 0x3fa3
: 0x3fa6A3      f0de  : 0x0001A5      0001  A6      0000  : 0x0000A8      0001  : 0x3434
: 0x000aA11     0001  : 0x3f0dA13     0001  A14     fc24  : 0x3fc7SAR     0008  : 0x0000
: 0x0000LBEG    0020  : 0x40cfLCOUNT  ffff  
Backtrace:0021:0x3fa3088b:0x3fa5 0x40d5fca008fa:0x3f2c

Core1 register dump:
: 0x40c7PS      0604  : 0x80c9A1      fce0  A2      fc88  : 0x3434A4      0004  : 0x3f0e
: 0x3f12A7      0001  : 0x0000A9      f0df  A10     0005  : 0x0000A12     080f  : 0x3f0b
: 0x0002A15     0000  : 0x0001EXCCAUSE0005  EXCVADDR0000  : 0x40ceLEND    0026  : 0xffff
 0x40c7fce0 0x40c9fcf008f4:0x3fc3 0x40d008413m 249s_oedm aecr upt ls..[mets Jun  8 2016 00:22:57

rst:0x8 (TG1WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:5628
load:0x40078000,len:7756
ho 0 tail 12 room 4
load:0x40080000,len:5948
0x40080000: _iram_start at /home/peter/esp/esp-idf/components/freertos/xtensa_vectors.S:1685

entry 0x40080318
0x40080318: _KernelExceptionVector at ??:?

W (31) boot: PRO CPU has been reset by WDT.
W (31) boot: WDT reset info: PRO CPU PC=0x40083306
0x40083306: spi_flash_disable_interrupts_caches_and_other_cpu at /home/peter/esp/esp-idf/components/spi_flash/cache_utils.c:135

Or sometimes like this:

Code: Select all

Interrupt wdt timeout $
n CPU0 PC      0021  : 0x0003A0      0892  : 0x3fa2
: 0x3f96A3      f0da  : 0x0001A5      0001  A6      0000  : 0x0000A8      0001  : 0x3430
: 0x000aA11     0001  : 0x3f0cA13     0001  A14     fc1c  : 0x3fc8SAR     0004  : 0x0000
: 0x0000LBEG    0020  : 0x40cfLCOUNT  ffff  
Backtrace:0021:0x3fa2089f:0x3fa4 0x40e1fc9008fe:0x000f

Core1 register dump:
: 0x40c7PS      0604  : 0x80c6A1      fcd0  A2      fc70  : 0x3430A4      0004  : 0x3f0d
: 0x3f01A7      0001  : 0x0000A9      f0db  A10     0005  : 0x0000A12     0813  : 0x3f0a
: 0x0002A15     0000  : 0x0001EXCCAUSE0005  EXCVADDR0000  : 0x40ceLEND    0026  : 0xffff
 0x40c7fcd0 0x40c5fce00808:0x3fc3 0x40dc008d[;1E(08 pcr_upCr upfahcni scrutd R=xfffisedo x[mCPU halted.
ets Jun  8 2016 00:22:57

rst:0x7 (TG0WDT_SYS_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:5612
load:0x40078000,len:7756
ho 0 tail 12 room 4
load:0x40080000,len:5876
0x40080000: _WindowOverflow4 at /home/peter/esp/esp-idf-v3.1/components/freertos/xtensa_vectors.S:1685

entry 0x40080318
0x40080318: _KernelExceptionVector at ??:?

W (31) boot: PRO CPU has been reset by WDT.
W (31) boot: WDT reset info: PRO CPU PC=0x4008fc06
0x4008fc06: commonErrorHandler at /home/peter/esp/esp-idf-v3.1/components/esp32/panic.c:649

W (31) boot: WDT reset info: APP CPU PC=0x4008fc46
0x4008fc46: panicHandler at /home/peter/esp/esp-idf-v3.1/components/esp32/panic.c:649

I (37) boot: ESP-IDF v3.1 2nd stage bootloader
I (42) boot: compile time 09:29:55
I (46) boot: Enabling RNG early entropy source...
I (51) boot: SPI Speed      : 40MHz
I (56) boot: SPI Mode       : DIO

Some characters are missing in the output above. I don't now why. This system is connected to a Raspberry Pi, maybe its not fast enough to keep up? I don't no at this point... it happens only during a crash, otherwise the Serial outputs correct/readable data.


Now according to the documentation, this is caused by the Interrupt Watchdog that kick in. So there might be a piece of code that loops infinitely. This I can understand.
But sometimes the reset reason is "TG0WDT_SYS_RESET" and sometimes its "TG1WDT_SYS_RESET". What to make of this. THe documentation says that TG0 is related to the "Task WDT" and the TG1 is related to the "Interrupt WDT".

Where to go from here? I have no clue where to start looking for the cause of this crash.
Anybody got some ideas?

[ESP_Sprite]

Can you decode the backtrace? If you run 'make monitor' as your serial terminal, it'll automatically do this.

[kluverp]

These outputs are from "make monitor".

I also dump the core to flash. So I can read it out later. It says different things, on different occasions, but this is the latest:

Code: Select all

================== CURRENT THREAD REGISTERS ===================
pc             0x400d41cf                                                                                                                                    0x400d41cf <heap_caps_init+23>
lbeg           0x0                                                                                                                                           0
lend           0x0                                                                                                                                           0
lcount         0x0                                                                                                                                           0
sar            0x0                                                                                                                                           0
ps             0x60130                                                                                                                                       393520
threadptr      <unavailable>
br             <unavailable>
scompare1      <unavailable>
acclo          <unavailable>
acchi          <unavailable>
m0             <unavailable>
m1             <unavailable>
m2             <unavailable>
m3             <unavailable>
expstate       <unavailable>
f64r_lo        <unavailable>
f64r_hi        <unavailable>
f64s           <unavailable>
fcr            <unavailable>
fsr            <unavailable>
a0             0x3ffc7270                                                                                                                                    1073508976
a1             0x0                                                                                                                                           0
a2             0x3ffc73a4                                                                                                                                    1073509284
a3             0x0                                                                                                                                           0
a4             0x0                                                                                                                                           0
a5             0x0                                                                                                                                           0
a6             0x0                                                                                                                                           0
a7             0x0                                                                                                                                           0
a8             0x0                                                                                                                                           0
a9             0x0                                                                                                                                           0
a10            0x0                                                                                                                                           0
a11            0x0                                                                                                                                           0
a12            0x0                                                                                                                                           0
a13            0x0                                                                                                                                           0
a14            0x0                                                                                                                                           0
a15            0x0                                                                                                                                           0

==================== CURRENT THREAD STACK =====================
#0  0x400d41cf in heap_caps_init () at /home/peter/esp/esp-idf-v3.1/components/heap/heap_caps_init.c:106

======================== THREADS INFO =========================
  Id   Target Id         Frame 
  7    process 6         0x4008c310 in xQueueGenericReceive (xQueue=0x3ffafebc, pvBuffer=0x0, xTicksToWait=4294967295, xJustPeeking=0) at /home/peter/esp/esp-idf-v3.1/components/freertos/queue.c:1445
  6    process 5         0x4008c310 in xQueueGenericReceive (xQueue=0x3ffaff10, pvBuffer=0x0, xTicksToWait=4294967295, xJustPeeking=0) at /home/peter/esp/esp-idf-v3.1/components/freertos/queue.c:1445
  5    process 4         0x4008c310 in xQueueGenericReceive (xQueue=0x3ffaea30, pvBuffer=0x0, xTicksToWait=4294967295, xJustPeeking=0) at /home/peter/esp/esp-idf-v3.1/components/freertos/queue.c:1445
  4    process 3         /mnt/pc/esp32/pi/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) [answered Y; input not from terminal]
/mnt/pc/esp32/pi/crosstool-NG/.build/src/gdb-7.10/gdb/inline-frame.c:171: internal-error: inline_frame_this_id: Assertion `!frame_id_eq (*this_id, outer_frame_id)' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n) [answered Y; input not from terminal]
GDB exited (None / )!
Problem occured! GDB exited, restart it.
Reading symbols from Binaries/2908V10.elf...done.
[New <main task>]
[New process 1]
[New process 2]
[New process 3]
[New process 4]
[New process 5]
[New process 6]
[Current thread is 1 (<main task>)]

======================= ALL MEMORY REGIONS ========================
Name   Address   Size   Attrs
.rtc.text 0x400c0000 0x0 RW  
.rtc_noinit 0x50000000 0x0 RW  
.iram0.vectors 0x40080000 0x400 R XA
.iram0.text 0x40080400 0x1149c RWXA
.dram0.data 0x3ffb0000 0x3b80 RW A
.noinit 0x3ffb3b80 0x0 RW  
.flash.rodata 0x3f400020 0x2f398 RW A
.flash.text 0x400d0018 0xab048 R XA
.coredump.tasks 0x3ffc733c 0x164 RW 
.coredump.tasks 0x3ffc71b0 0x184 RW 
.coredump.tasks 0x3ffc7e98 0x164 RW 
.coredump.tasks 0x3ffc7d20 0x170 RW 
.coredump.tasks 0x3ffc792c 0x164 RW 
.coredump.tasks 0x3ffc77b0 0x174 RW 
.coredump.tasks 0x3ffc88f8 0x164 RW 
.coredump.tasks 0x3ffc87a0 0x150 RW 
.coredump.tasks 0x3ffafa88 0x164 RW 
.coredump.tasks 0x3ffaf900 0x180 RW 
.coredump.tasks 0x3ffc5c5c 0x164 RW 
.coredump.tasks 0x3ffc5ad0 0x184 RW 
.coredump.tasks 0x3ffc56f0 0x164 RW 
.coredump.tasks 0x3ffc5560 0x188 RW