### Hardware
- Board: ESP32-S3 Korvo-2 V3.1
Verified hardware is working correctly (as the official ADF AEC example on the same board works correctly with AEC)
- Receives audio data from a server and plays it through the speaker
Simultaneously captures microphone audio and sends it back to the server
Before adding AEC, the basic TX/RX audio path was functioning.
- Created two pipelines:
- One for recording
One for playback
Used this reference:
[https://github.com/espressif/esp-adf/bl ... examples.c]
Logs show no critical errors
Pipeline linking and initialization appear correct - One for recording
After integrating AEC:
- The audio sent to the server contains only noise
No audio is heard from the speaker
No obvious runtime errors in logs
I would appreciate guidance on:
- Correct integration of AEC into a custom full-duplex pipeline
Proper feed/fetch usage and frame size alignment
Any common pitfalls when combining custom networking logic with AEC
- Pipeline configuration
AEC/AFE initialization config
I2S settings
Code: Select all
I (8665) AUDIO_AEC: [2.0] Initialize I2S Stream Writer (Playback)
W (8675) i2s_common: dma frame num is out of dma buffer size, limited to 1023
I (8675) AUDIO_AEC: [1.0] Initializing Audio Board...
W (8685) i2c_bus_v2: I2C master handle is NULL, will create new one
I (8695) gpio: GPIO[17]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (8705) gpio: GPIO[18]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (8715) DRV8311: ES8311 in Slave mode
I (8725) gpio: GPIO[48]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
I (8725) DRV8311: CFG I2S: mode = 3, bits = 1, fmt = 0, samples = 7
I (8745) ES7210: ES7210 in Slave mode
I (8755) ES7210: Enable ES7210_INPUT_MIC1
I (8755) ES7210: Enable ES7210_INPUT_MIC2
I (8755) ES7210: Enable ES7210_INPUT_MIC3
W (8765) ES7210: Enable TDM mode. ES7210_SDP_INTERFACE2_REG12: 2
I (8765) ES7210: config fmt 60
I (8765) AUDIO_HAL: Codec mode is 3, Ctrl:1
I (8775) AUDIO_AEC: ES7210 ADC gains configured
I (8775) AUDIO_AEC: [2.1] Initialize I2S Stream Reader (Recording)
I (8785) AUDIO_AEC: [3.2] Create Recording Pipeline
I (8785) AUDIO_AEC: [2.2] Initialize Raw Stream
I (8795) AUDIO_AEC: [3.1] Create AEC Algorithm Stream
E (8805) MODEL_LOADER: Can not find model in partition table
E (8805) ALGORITHM_STREAM: Failed to load models
I (8815) RAW_OPUS_ENC: Raw Opus encoder init
I (8815) AUDIO_AEC: [3.3] Register elements to recording pipeline
I (8825) AUDIO_AEC: [3.4] Link elements [aec]-->[opus]-->[raw_sink]
I (8835) AUDIO_PIPELINE: link el->rb, el:0x3c181e04, tag:aec, rb:0x3c1820d4
I (8835) AUDIO_PIPELINE: link el->rb, el:0x3c181f60, tag:opus, rb:0x3c18221c
I (8845) AUDIO_AEC: AEC initialized successfully
I (8855) AUDIO_AEC: [4.1] Create Playback Pipeline
I (8855) AUDIO_AEC: [4.2] Initialize Raw Write Stream (Source)
I (8865) AUDIO_AEC: [4.3] Initialize OPUS Decoder Element
audio_pipeline_link handle 0x3c182a50
I (8875) AUDIO_AEC: [4.4] Register elements to playback pipeline
iam here 1
iam here 2
iam here 3
I (8885) AUDIO_AEC: [4.5] Link elements [raw_read]-->decoder-->[filter]-->[i2s]
I (8895) AUDIO_PIPELINE: link el->rb, el:0x3c182a84, tag:raw_read, rb:0x3c182f58
I (8905) AUDIO_PIPELINE: link el->rb, el:0x3c182c60, tag:decoder, rb:0x3c184fa0
I (8905) AUDIO_PIPELINE: link el->rb, el:0x3c182dd4, tag:filter, rb:0x3c1857e8
I (8915) AUDIO_AEC: [5.1] Set up event listener
I (8925) AUDIO_AEC: [5.2] Listening event from all elements of pipeline
I (8925) AUDIO_AEC: [5.3] Listening event from peripherals
I (8935) AUDIO_AEC: Audio codec initialization complete
I (9285) wifi:<ba-add>idx:0 (ifx:0, a6:79:b0:8a:0a:65), tid:0, ssn:3, winSize:64
I (9335) wifi:<ba-add>idx:1 (ifx:0, a6:79:b0:8a:0a:65), tid:4, ssn:1, winSize:64
W (9695) wifi:exceed max band, 2g, ngroup:5
W (10925) wifi:exceed max band, 2g, ngroup:5
W (11235) wifi:exceed max band, 2g, ngroup:5
W (11955) wifi:exceed max band, 2g, ngroup:5
W (12155) wifi:exceed max band, 2g, ngroup:5
INFO /Users/karanrajput/esp-adf/examples/iot-sdk-adf/src/dtls_srtp.c 331 Created inbound SRTP session
INFO /Users/karanrajput/esp-adf/examples/iot-sdk-adf/src/dtls_srtp.c 352 Created outbound SRTP session
I (12615) videosdk: Generated displayName: 8hwvh9Xh
g_codec 2
W (13285) wifi:exceed max band, 2g, ngroup:5
W (14205) wifi:exceed max band, 2g, ngroup:5
W (14405) wifi:exceed max band, 2g, ngroup:5
W (15225) wifi:exceed max band, 2g, ngroup:5
INFO /Users/karanrajput/esp-adf/examples/iot-sdk-adf/src/dtls_srtp.c 331 Created inbound SRTP session
INFO /Users/karanrajput/esp-adf/examples/iot-sdk-adf/src/dtls_srtp.c 352 Created outbound SRTP session
W (15745) wifi:exceed max band, 2g, ngroup:5
I (15945) AUDIO_AEC: Peer connected, starting pipelines
I (15945) AUDIO_ELEMENT: [raw_read-0x3c182a84] Element task created
I (15945) AUDIO_THREAD: The decoder task allocate stack on external memory
I (15955) AUDIO_ELEMENT: [decoder-0x3c182c60] Element task created
I (15955) AUDIO_THREAD: The filter task allocate stack on external memory
I (15965) AUDIO_ELEMENT: [filter-0x3c182dd4] Element task created
I (15975) AUDIO_ELEMENT: [i2s-0x3c180fac] Element task created
I (15975) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8183448 Bytes, Inter:49187 Bytes, Dram:49187 Bytes, Dram largest free:18432Bytes
I (15995) AUDIO_ELEMENT: [decoder] AEL_MSG_CMD_RESUME,state:1
I (15995) OPUS_DECODER: OPUS codec
I (16005) CODEC_ELEMENT_HELPER: The element is 0x3c182c60. The reserve data 2 is 0x0.
I (16015) OPUS_DECODER: a new song playing
I (16015) AUDIO_ELEMENT: [filter] AEL_MSG_CMD_RESUME,state:1
I (16025) RSP_FILTER: sample rate of source data : 16000, channel of source data : 1, sample rate of destination data : 16000, channel of destination data : 2
I (16025) AUDIO_PIPELINE: Pipeline started
I (16045) AUDIO_THREAD: The aec task allocate stack on external memory
I (16045) AUDIO_ELEMENT: [aec-0x3c181e04] Element task created
I (16055) AUDIO_THREAD: The opus task allocate stack on external memory
I (16065) AUDIO_ELEMENT: [opus-0x3c181f60] Element task created
I (16065) AUDIO_ELEMENT: [raw_sink-0x3c181c90] Element task created
I (16075) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:8107424 Bytes, Inter:48499 Bytes, Dram:48499 Bytes, Dram largest free:18432Bytes
I (16095) AUDIO_ELEMENT: [aec] AEL_MSG_CMD_RESUME,state:1
W (16095) AFE_CONFIG: For single microphone channel, SE is deactivated.
I (16135) AFE: AFE Version: (1MIC_V250121)
I (16135) AFE: Input PCM Config: total 2 channels(1 microphone, 1 playback), sample rate:16000
I (16135) AFE: AFE Pipeline: [input] -> |AEC(VOIP_LOW_COST)| -> |NS(WebRTC)| -> [output]
I (16145) AUDIO_THREAD: The algo_fetch task allocate stack on external memory
I (16155) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16165) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16185) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16205) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16225) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16245) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16255) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16275) AUDIO_AEC: i2s_read_cb: Read 1024 bytes
I (16295) AUDIO_AEC: i2s_read_cb: Read 1024 bytes