ESP32-S3 Crash During Wake Word Model Initialization (LoadProhibited Exception)

[abdulbaseer]

issue:(!pls help I have been trying to solve this since 2 week)

Using ESP32-S3 with ESP-IDF v5.5 and esp-sr library v2.0.

Speech recognition models stored in a flash partition labeled "model" at offset 0x410000, size 8MB.

Partition is successfully found and memory-mapped.

Model data pointer (wn_data->data[0]) appears valid and outputs correct hex data.

Crash occurs immediately during wakenet->create() call inside esp-sr library (wakenet9s_quantized_mc.c line 202).

Exception: LoadProhibited (memory access violation).

Backtrace points to model_destroy() function during model initialization.

Application reboots after crash.








Summary of ESP32-S3 Wake Word Detection Crash Issue

Environment & Setup
Device: ESP32-S3

ESP-IDF Version: v5.5-dirty (built Jul 28, 2025)

esp-sr (Speech Recognition) library version: v2.0.0 (latest)

Flash Size: 16 MB external SPI flash

Partition Table:

nvs: 0x9000, size 0x4000

otadata: 0xd000, size 0x2000

app0 (factory app): 0x10000, size 4MB

model (custom data): 0x410000, size 8MB

Description
The project uses a multinet + wakenet speech model approach for wake word detection and command recognition.

Models are stored in the "model" partition, loaded via esp_srmodel_init("model") which uses flash memory mmap to map the partition.

The partition is successfully mmap’d, and initial logs indicate models are loaded correctly:

yaml
Copy
Edit
I (1978) MODEL_LOADER: The storage free size is 31104 KB
I (1978) MODEL_LOADER: The partition size is 8192 KB
I (1978) MODEL_LOADER: Successfully load srmodels
I (1988) WakeMultinet: Using WakeNet: wn9s_nihaoxiaozhi
I (1988) WakeMultinet: Model data size: 124545
Hex dump of model data pointer looks valid (shows expected data bytes).

Problem
The crash happens immediately during call to:

c
Copy
Edit
wn_handle = wakenet->create((const void *)wn_data->data[0], WAKE_MODE);


esp_log:

Rebooting...
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375b60
--- 0x40375b60: esp_restart_noos at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/components/esp_system/port/soc/esp32s3/system_internal.c:162
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce2820,len:0x159c
load:0x403c8700,len:0xd24
--- 0x403c8700: _stext at ??:?
load:0x403cb700,len:0x2f48
entry 0x403c8924
--- 0x403c8924: call_start_cpu0 at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/components/bootloader/subproject/main/bootloader_start.c:25
I (24) boot: ESP-IDF v5.5-dirty 2nd stage bootloader
I (24) boot: compile time Jul 28 2025 14:52:56
I (24) boot: Multicore bootloader
I (25) boot: chip revision: v0.2
I (27) boot: efuse block revision: v1.3
I (31) boot.esp32s3: Boot SPI Speed : 80MHz
I (35) boot.esp32s3: SPI Mode : DIO
I (39) boot.esp32s3: SPI Flash Size : 16MB
I (42) boot: Enabling RNG early entropy source...
I (47) boot: Partition Table:
I (49) boot: ## Label Usage Type ST Offset Length
I (56) boot: 0 nvs WiFi data 01 02 00009000 00004000
I (62) boot: 1 otadata OTA data 01 00 0000d000 00002000
I (69) boot: 2 app0 factory app 00 00 00010000 00400000
I (75) boot: 3 model Unknown data 01 06 00410000 00800000
I (82) boot: End of partition table
I (85) boot: Defaulting to factory image
I (89) esp_image: segment 0: paddr=00010020 vaddr=3c0e0020 size=b60a0h (745632) map
I (228) esp_image: segment 1: paddr=000c60c8 vaddr=3fc9ba00 size=06e3ch ( 28220) load
I (234) esp_image: segment 2: paddr=000ccf0c vaddr=40374000 size=0310ch ( 12556) load
I (237) esp_image: segment 3: paddr=000d0020 vaddr=42000020 size=dde0ch (908812) map
I (399) esp_image: segment 4: paddr=001ade34 vaddr=4037710c size=148c0h ( 84160) load
I (418) esp_image: segment 5: paddr=001c26fc vaddr=600fe000 size=00020h ( 32) load
I (428) boot: Loaded app from partition at offset 0x10000
I (428) boot: Disabling RNG early entropy source...
I (438) cpu_start: Multicore app
I (447) cpu_start: Pro cpu start user code
I (447) cpu_start: cpu freq: 160000000 Hz
I (447) app_init: Application information:
I (448) app_init: Project name: esp32_assistant
I (452) app_init: App version: 1
I (455) app_init: Compile time: Jul 28 2025 14:19:53
I (460) app_init: ELF file SHA256: 7ff66f115...
I (465) app_init: ESP-IDF: v5.5-dirty
I (469) efuse_init: Min chip rev: v0.0
I (473) efuse_init: Max chip rev: v0.99
I (477) efuse_init: Chip rev: v0.2
I (481) heap_init: Initializing. RAM available for dynamic allocation:
I (487) heap_init: At 3FCA6FF8 len 00042718 (265 KiB): RAM
I (492) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (497) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (502) heap_init: At 600FE020 len 00001FC8 (7 KiB): RTCRAM
I (509) spi_flash: detected chip: generic
I (511) spi_flash: flash io: dio
I (515) sleep_gpio: Configure to isolate all GPIO pins in sleep state
I (521) sleep_gpio: Enable automatic switching of GPIO sleep configuration
I (528) main_task: Started on CPU0
I (538) main_task: Calling app_main()
I (538) MAIN: Starting application
I (558) pp: pp rom version: e7ae62f
I (558) net80211: net80211 rom version: e7ae62f
I (568) wifi:wifi driver task: 3fcabde8, prio:23, stack:6656, core=0
I (578) wifi:wifi firmware version: f3dbad7
I (578) wifi:wifi certification version: v7.0
I (578) wifi:config NVS flash: enabled
I (578) wifi:config nano formatting: disabled
I (578) wifi:Init data frame dynamic rx buffer num: 32
I (588) wifi:Init static rx mgmt buffer num: 5
I (588) wifi:Init management short buffer num: 32
I (588) wifi:Init dynamic tx buffer num: 32
I (598) wifi:Init static tx FG buffer num: 2
I (598) wifi:Init static rx buffer size: 1600
I (608) wifi:Init static rx buffer num: 10
I (608) wifi:Init dynamic rx buffer num: 32
I (608) wifi_init: rx ba win: 6
I (618) wifi_init: accept mbox: 6
I (618) wifi_init: tcpip mbox: 32
I (618) wifi_init: udp mbox: 6
I (628) wifi_init: tcp mbox: 6
I (628) wifi_init: tcp tx win: 5760
I (628) wifi_init: tcp rx win: 5760
I (628) wifi_init: tcp mss: 1440
I (638) wifi_init: WiFi IRAM OP enabled
I (638) wifi_init: WiFi RX IRAM OP enabled
I (648) phy_init: phy_version 701,f4f1da3a,Mar 3 2025,15:50:10
I (698) wifi:mode : sta (30:ed:a0:25:fd:b0)
I (698) wifi:enable tsf
I (698) wifi_sta: Wi-Fi STA Init Finished
I (698) MQTT_CLIENT: Other MQTT event id:7
E (698) esp-tls: [sock=54] connect() error: Host is unreachable
E (698) transport_base: Failed to open a new connection: 32772
E (708) mqtt_client: Error transport connect
E (708) MQTT_CLIENT: MQTT error occurred
I (718) MQTT_CLIENT: Disconnected from MQTT broker
I (718) MAIN: Starting audio recording...
I (728) MIC_I2S: I2S STD RX channel initialized @16kHz mono
I (1988) MODEL_LOADER: The storage free size is 31104 KB
I (1988) MODEL_LOADER: The partition size is 8192 KB
I (1988) MODEL_LOADER: Successfully load srmodels
I (1988) WakeMultinet: Using WakeNet: wn9s_nihaoxiaozhi
I (1998) WakeMultinet: Model data size: 124545
I (1998) WakeMultinet: 50 00 00 00 01 00 00 00 01 00 00 00 00 00 00 00
I (2008) WakeMultinet: f0 ff ff ff 05 01 ec f9 75 f6 74 e0 12 12 be f0
I (2018) WakeMultinet: 10 05 be 09 de ea 29 02 7d 06 e3 ea af 00 31 e2
I (2018) WakeMultinet: 40 19 c0 04 34 14 8c 0c 8e ca a4 01 95 18 e7 15
I (2028) WakeMultinet: 5a 09 d8 b9 40 06 18 0d 7e fd a9 03 43 f9 60 fc
I (2038) WakeMultinet: 80 fc d9 f9 d4 03 18 00 5e fd 76 ed 85 f3 72 0b
I (2038) WakeMultinet: 1d 03 aa 02 6b f7 87 fd 04 01 6e 13 6a f3 6c e4
I (2048) WakeMultinet: e9 f5 3f 0a 31 0c ad 0f f2 f9 3f fa 40 01 70 f1
Guru Meditation Error: Core 0 panic'ed (InstrFetchProhibited). Exception was unhandled.

Core 0 register dump:
PC : 0x00000000 PS : 0x00060130 A0 : 0x8202246c A1 : 0x3fcaa8e0
A2 : 0x3fcb8008 A3 : 0x3c44257c A4 : 0x3c0ecb04 A5 : 0x00000800
A6 : 0x3c0e5c08 A7 : 0x3fcaaa90 A8 : 0x82020e3c A9 : 0x00000050
A10 : 0x3fcb8000 A11 : 0x3fcaaa20 A12 : 0x3fcaaa00 A13 : 0x0000000c
A14 : 0x3fcaa940 A15 : 0x000000a8 SAR : 0x0000000f EXCCAUSE: 0x00000014
EXCVADDR: 0x00000000 LBEG : 0x400556d5 LEND : 0x400556e5 LCOUNT : 0xfffffff3
--- 0x400556d5: strlen in ROM
--- 0x400556e5: strlen in ROM


Backtrace: 0xfffffffd:0x3fcaa8e0 0x42022469:0x3fcaa900 0x42022480:0x3fcaaac0 0x4200c694:0x3fcaaae0 0x4200c239:0x3fcaab10 0x420dcfa0:0x3fcaab50 0x4037ddc5:0x3fcaab80
--- 0x42022469: model_create at /home/sunxiangyu/workspace/esp_sr_lib/components/wakenet/wakenet9s_quantized_mc.c:512
--- 0x42022480: model_init at /home/sunxiangyu/workspace/esp_sr_lib/components/wakenet/wakenet9s_quantized_mc.c:904
--- 0x4200c694: wakeword_init_multinet at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/components/custom_audio/wakeword.c:75
--- 0x4200c239: app_main at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/main/main.c:94
--- 0x420dcfa0: main_task at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/components/freertos/app_startup.c:208
--- 0x4037ddc5: vPortTaskWrapper at F:/personal/internship/esp32-voice-assistant/firmware_SR/esp32_chatbot/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:139




ELF file SHA256: 7ff66f115





code(relevant portion)
#include "esp_wn_models.h"
#include "esp_mn_models.h"
#include "wakeword.h"
#include "mic_i2s.h"
#include "esp_log.h"
#include "model_path.h" // Replace esp_srmodel.h with model_path.h
#include "esp_wn_iface.h"
#include "esp_mn_iface.h"
#include "esp_mn_speech_commands.h"
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"

static const char *TAG = "WakeMultinet";

#define SAMPLE_RATE 16000
#define BUFFER_SIZE 512
#define WAKE_MODE DET_MODE_95
#define CMD_TIMEOUT 40 // ~0.5s @ 16kHz

static wakeword_callback_t ww_callback = NULL;
static command_callback_t cmd_callback = NULL;

static srmodel_list_t *sr_models = NULL;
static const esp_wn_iface_t *wakenet = NULL;
static const esp_mn_iface_t *multinet = NULL;
static model_iface_data_t *wn_handle = NULL;
static model_iface_data_t *mn_handle = NULL;

void wakeword_init_multinet(wakeword_callback_t wake_cb, command_callback_t command_cb) {
ww_callback = wake_cb;
cmd_callback = command_cb;

// Initialize I2S microphone!!!!!!!!!!DO NOT USE HERE AS ALREADY USED IN MAIN
//i2s_mic_init();!!!!!!! will cause double initialization error
/*
W (1987) i2s_platform: i2s controller 1 has been occupied by i2s_driver
E (1987) i2s_common: i2s_new_channel(996): no available channel found
*/

// Initialize model list from partition
sr_models = esp_srmodel_init("model");
if (!sr_models) {
ESP_LOGE(TAG, "Failed to init model list");
return;
}

// Initialize WakeNet
char *wn_name = esp_srmodel_filter(sr_models, ESP_WN_PREFIX, NULL);
if (!wn_name) {
ESP_LOGE(TAG, "No WakeNet model found");
esp_srmodel_deinit(sr_models);
return;
}

int wn_index = esp_srmodel_exists(sr_models, wn_name);
if (!wn_name) {
ESP_LOGE(TAG, "No WakeNet model found");
esp_srmodel_deinit(sr_models);
return;
}

srmodel_data_t *wn_data = sr_models->model_data[wn_index];

wakenet = esp_wn_handle_from_name(wn_name);
if (!wakenet) {
ESP_LOGE(TAG, "Failed to get WakeNet handle for %s", wn_name);
esp_srmodel_deinit(sr_models);
return;
}

//for debugging
ESP_LOGI(TAG, "Using WakeNet: %s", wn_name);
ESP_LOGI(TAG, "Model data size: %d", wn_data->sizes[0]);
ESP_LOG_BUFFER_HEX(TAG, wn_data->data[0], 128);
wn_handle = wakenet->create((const void *)wn_data->data[0], WAKE_MODE);
if (!wn_handle) {
ESP_LOGE(TAG, "WakeNet creation failed for %s", wn_name);
esp_srmodel_deinit(sr_models);
return;
}

// Initialize MultiNet
char *mn_name = esp_srmodel_filter(sr_models, ESP_MN_PREFIX, "en"); // or "cn" based on your model
if (!mn_name) {
ESP_LOGE(TAG, "No MultiNet model found");
wakenet->destroy(wn_handle);
esp_srmodel_deinit(sr_models);
return;
}

multinet = esp_mn_handle_from_name(mn_name);
if (!multinet) {
ESP_LOGE(TAG, "Failed to get MultiNet handle for %s", mn_name);
wakenet->destroy(wn_handle);
esp_srmodel_deinit(sr_models);
return;
}

mn_handle = multinet->create(mn_name,6000);
if (!mn_handle) {
ESP_LOGE(TAG, "MultiNet creation failed for %s", mn_name);
wakenet->destroy(wn_handle);
esp_srmodel_deinit(sr_models);
return;
}

// Configure MultiNet commands
char *lang = multinet->get_language(mn_handle);
esp_mn_commands_clear();
if (strcmp(lang, ESP_MN_CHINESE) == 0) {
esp_mn_commands_add(1, "da kai kong tiao");
esp_mn_commands_add(2, "guan bi kong tiao");
} else if (strcmp(lang, ESP_MN_ENGLISH) == 0) {
esp_mn_commands_add(1, "turn on the light");
esp_mn_commands_add(2, "turn off the light");
} else {
ESP_LOGE(TAG, "Invalid language: %s", lang);
wakenet->destroy(wn_handle);
multinet->destroy(mn_handle);
esp_srmodel_deinit(sr_models);
return;
}
esp_mn_error_t *error_phrases = esp_mn_commands_update();
if (error_phrases) {
ESP_LOGE(TAG, "Failed to update commands");
wakenet->destroy(wn_handle);
multinet->destroy(mn_handle);
esp_srmodel_deinit(sr_models);
return;
}
multinet->print_active_speech_commands(mn_handle);

ESP_LOGI(TAG, "WakeNet (%s) and MultiNet (%s) initialized", wn_name, mn_name);
}

void wakeword_task(void *arg) {
int16_t *buffer = calloc(BUFFER_SIZE, sizeof(int16_t));
if (!buffer) {
ESP_LOGE(TAG, "Buffer allocation failed");
vTaskDelete(NULL);
return;
}

while (1) {
int bytes_read = i2s_mic_read((char *)buffer, BUFFER_SIZE * sizeof(int16_t));
if (bytes_read <= 0) continue;

if (wakenet->detect(wn_handle, buffer) == WAKENET_DETECTED) {
ESP_LOGI(TAG, "Wake word detected");
if (ww_callback) ww_callback();

for (int i = 0; i < CMD_TIMEOUT; ++i) {
bytes_read = i2s_mic_read((char *)buffer, BUFFER_SIZE * sizeof(int16_t));
if (bytes_read <= 0) continue;

esp_mn_state_t mn_state = multinet->detect(mn_handle, buffer);
if (mn_state == ESP_MN_STATE_DETECTED) {
esp_mn_results_t *res = multinet->get_results(mn_handle);
if (res && res->num > 0) {
ESP_LOGI(TAG, "Command recognized: %s (ID: %d)", res->string, res->command_id[0]);
if (cmd_callback) cmd_callback(res->string);
multinet->clean(mn_handle);
break;
}
} else if (mn_state == ESP_MN_STATE_TIMEOUT) {
ESP_LOGI(TAG, "Command detection timeout");
break;
}
}
}
}

// Cleanup (not usually reached)
free(buffer);
wakenet->destroy(wn_handle);
multinet->destroy(mn_handle);
esp_srmodel_deinit(sr_models);
vTaskDelete(NULL);
}