I2S Stream seems to be one bit short

[danyo93]

Hi all,

I am currently playing around with ESP-ADF using an ESP32 Dev-Board together with a Wolfson WM8731 Audio Codec Dev-Board https://www.digikey.de/product-detail/d ... ND/4495592. I added my own driver lib for the codec as well as a board description and as for the configuration via I2C, I got everything running. Now I wanted to try to execute the the play_mp3_control example (minus the control) and play the mp3 sample through some headphones plugged into the WM8731. Here is the source code of the program:

Code: Select all

void app_main() {

    wifi_sta_start();
    syncTSF_init();
    syncTSF_start();

    audio_pipeline_handle_t pipeline;
    audio_element_handle_t mp3_decoder, i2s_stream_writer;

    ESP_LOGI(TAG, "[ 1 ] Start audio codec chip");
    audio_board_handle_t board_handle = audio_board_init();
    audio_hal_ctrl_codec(board_handle->audio_hal, AUDIO_HAL_CODEC_MODE_BOTH, AUDIO_HAL_CTRL_START);

    ESP_LOGI(TAG, "[ 2 ] Create audio pipeline, add all elements to pipeline, and subscribe pipeline event");
    audio_pipeline_cfg_t pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG();
    pipeline = audio_pipeline_init(&pipeline_cfg);
    mem_assert(pipeline);

    ESP_LOGI(TAG, "[2.1] Create mp3 decoder to decode mp3 file and set custom read callback");
    mp3_decoder_cfg_t mp3_cfg = DEFAULT_MP3_DECODER_CONFIG();
    mp3_decoder = mp3_decoder_init(&mp3_cfg);
    audio_element_set_read_cb(mp3_decoder, mp3_music_read_cb, NULL);

    ESP_LOGI(TAG, "[2.2] Create i2s stream to write data to ESP32 internal DAC");
    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
    i2s_cfg.type = AUDIO_STREAM_WRITER;
    i2s_stream_writer = i2s_stream_init(&i2s_cfg);

    ESP_LOGI(TAG, "[2.3] Register all elements to audio pipeline");
    audio_pipeline_register(pipeline, mp3_decoder, "mp3");
    audio_pipeline_register(pipeline, i2s_stream_writer, "i2s");

    ESP_LOGI(TAG, "[2.4] Link it together [mp3_music_read_cb]-->mp3_decoder-->i2s_stream-->[ESP32 DAC]");
    const char *link_tag[2] = {"mp3", "i2s"};
    audio_pipeline_link(pipeline, &link_tag[0], 2);

    ESP_LOGI(TAG, "[ 3 ] Set up  event listener");
    audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
    audio_event_iface_handle_t evt = audio_event_iface_init(&evt_cfg);

    ESP_LOGI(TAG, "[3.1] Listening event from all elements of pipeline");
    audio_pipeline_set_listener(pipeline, evt);

    ESP_LOGI(TAG, "[ 4 ] Start audio_pipeline");
    audio_pipeline_run(pipeline);

    while (1) {
        audio_event_iface_msg_t msg;
        esp_err_t ret = audio_event_iface_listen(evt, &msg, portMAX_DELAY);
        if (ret != ESP_OK) {
            ESP_LOGE(TAG, "[ * ] Event interface error : %d", ret);
            continue;
        }

        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT && msg.source == (void *) mp3_decoder
            && msg.cmd == AEL_MSG_CMD_REPORT_MUSIC_INFO) {
            audio_element_info_t music_info = {0};
            audio_element_getinfo(mp3_decoder, &music_info);

            ESP_LOGI(TAG, "[ * ] Receive music info from mp3 decoder, sample_rates=%d, bits=%d, ch=%d",
                     music_info.sample_rates, music_info.bits, music_info.channels);

            audio_element_setinfo(i2s_stream_writer, &music_info);
            i2s_stream_set_clk(i2s_stream_writer, music_info.sample_rates, music_info.bits, music_info.channels);
            continue;
        }
        /* Stop when the last pipeline element (i2s_stream_writer in this case) receives stop event */
        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT && msg.source == (void *) i2s_stream_writer
            && msg.cmd == AEL_MSG_CMD_REPORT_STATUS
            && (((int)msg.data == AEL_STATUS_STATE_STOPPED) || ((int)msg.data == AEL_STATUS_STATE_FINISHED))) {
            break;
        }
    }

    ESP_LOGI(TAG, "[ 5 ] Stop audio_pipeline");
    audio_pipeline_stop(pipeline);
    audio_pipeline_wait_for_stop(pipeline);
    audio_pipeline_terminate(pipeline);

    audio_pipeline_unregister(pipeline, mp3_decoder);
    audio_pipeline_unregister(pipeline, i2s_stream_writer);

    /* Terminate the pipeline before removing the listener */
    audio_pipeline_remove_listener(pipeline);

    /* Make sure audio_pipeline_remove_listener is called before destroying event_iface */
    audio_event_iface_destroy(evt);

    /* Release all resources */
    audio_pipeline_deinit(pipeline);
    audio_element_deinit(i2s_stream_writer);
    audio_element_deinit(mp3_decoder);
}

And - just to be thorough - here is the console output:

Code: Select all

␛[0;32mI (1678) MAIN: [ 1 ] Start audio codec chip␛[0m
␛[0;32mI (1688) WM8731: Starting I2C Software Control Interface...␛[0m
␛[0;32mI (1698) WM8731: ...Uploading Standard Configuration...␛[0m
␛[0;32mI (1708) WM8731: OUTPUT_REG [0] set to 0x5F9␛[0m
␛[0;32mI (1708) WM8731: OUTPUT_REG [1] set to 0x7F9␛[0m
␛[0;32mI (1708) WM8731: ...Done!␛[0m
␛[0;32mI (1708) AUDIO_HAL: Codec mode is 3, Ctrl:1␛[0m
␛[0;32mI (1718) WM8731: LIN_IN_LEFT: 117␛[0m
␛[0;32mI (1718) WM8731: LIN_IN_RIGHT: 317␛[0m
␛[0;32mI (1728) WM8731: HEADPHONE_LEFT: 5F9␛[0m
␛[0;32mI (1728) WM8731: HEADPHONE_RIGHT: 7F9␛[0m
␛[0;32mI (1728) WM8731: ANALOG_AUDIO_PATH: 815␛[0m
␛[0;32mI (1738) WM8731: DIGITAL_AUDIO_PATH: A10␛[0m
␛[0;32mI (1738) WM8731: POWER_REG: C60␛[0m
␛[0;32mI (1748) WM8731: DIGITAL_INTERFACE: E0A␛[0m
␛[0;32mI (1748) WM8731: SAMPLING_CTRL: 1002␛[0m
␛[0;32mI (1758) WM8731: ACTIVE_CTRL: 3001␛[0m
␛[0;32mI (1758) MAIN: [ 2 ] Create audio pipeline, add all elements to pipeline, and subscribe pipeline event␛[0m
␛[0;32mI (1768) MAIN: [2.1] Create mp3 decoder to decode mp3 file and set custom read callback␛[0m
␛[0;32mI (1778) MP3_DECODER: MP3 init␛[0m
␛[0;32mI (1778) MAIN: [2.2] Create i2s stream to write data to ESP32 internal DAC␛[0m
␛[0;32mI (1788) I2S: DMA Malloc info, datalen=blocksize=2048, dma_buf_count=8␛[0m
␛[0;32mI (1798) I2S: DMA Malloc info, datalen=blocksize=2048, dma_buf_count=8␛[0m
␛[0;32mI (1828) I2S: APLL: Req RATE: 48000, real rate: 48000.016, BITS: 16, CLKM: 1, BCK_M: 12, MCLK: 18432006.000, SCLK: 2304000.750000, diva: 1, divb: 0␛[0m
␛[0;32mI (1828) MOJO_TX_V1: I2S0, MCLK output by GPIO0␛[0m
␛[0;32mI (1838) MAIN: [2.3] Register all elements to audio pipeline␛[0m
␛[0;32mI (1838) MAIN: [2.4] Link it together [mp3_music_read_cb]-->mp3_decoder-->i2s_stream-->[ESP32 DAC]␛[0m
␛[0;32mI (1848) AUDIO_PIPELINE: link el->rb, el:0x3ffc6110, tag:mp3, rb:0x3ffcecbc␛[0m
␛[0;32mI (1858) MAIN: [ 3 ] Set up  event listener␛[0m
␛[0;32mI (1858) MAIN: [3.1] Listening event from all elements of pipeline␛[0m
␛[0;32mI (1868) MAIN: [ 4 ] Start audio_pipeline␛[0m
␛[0;32mI (1878) AUDIO_ELEMENT: [mp3-0x3ffc6110] Element task created␛[0m
␛[0;32mI (1878) AUDIO_ELEMENT: [i2s-0x3ffce960] Element task created␛[0m
␛[0;32mI (1888) AUDIO_PIPELINE: Func:audio_pipeline_run, Line:359, MEM Total:184868 Bytes
␛[0m
␛[0;32mI (1898) AUDIO_ELEMENT: [mp3] AEL_MSG_CMD_RESUME,state:1␛[0m
␛[0;32mI (1898) MP3_DECODER: MP3 opened␛[0m
␛[0;32mI (1908) CODEC_ELEMENT_HELPER: The element is 0x3ffc6110. The reserve data 2 is 0x0.␛[0m
␛[0;32mI (1918) AUDIO_ELEMENT: [i2s] AEL_MSG_CMD_RESUME,state:1␛[0m
␛[0;32mI (1928) I2S_STREAM: AUDIO_STREAM_WRITER␛[0m
␛[0;32mI (1948) AUDIO_PIPELINE: Pipeline started␛[0m
␛[0;32mI (1948) MAIN: [ * ] Receive music info from mp3 decoder, sample_rates=48000, bits=16, ch=2␛[0m
␛[0;32mI (2038) AUDIO_ELEMENT: [i2s] AEL_MSG_CMD_PAUSE␛[0m
␛[0;32mI (2068) I2S: APLL: Req RATE: 48000, real rate: 48000.016, BITS: 16, CLKM: 1, BCK_M: 12, MCLK: 18432006.000, SCLK: 2304000.750000, diva: 1, divb: 0␛[0m
␛[0;32mI (2068) AUDIO_ELEMENT: [i2s] AEL_MSG_CMD_RESUME,state:4␛[0m
␛[0;32mI (2078) I2S_STREAM: AUDIO_STREAM_WRITER␛[0m
␛[0;32mI (8508) AUDIO_ELEMENT: IN-[mp3] AEL_IO_DONE,-2␛[0m
␛[0;32mI (8648) MP3_DECODER: Closed␛[0m
␛[0;32mI (8668) AUDIO_ELEMENT: IN-[i2s] AEL_IO_DONE,-2␛[0m
␛[0;32mI (8838) MAIN: [ 5 ] Stop audio_pipeline␛[0m
␛[0;31mE (8838) AUDIO_ELEMENT: [mp3] Element already stopped␛[0m
␛[0;31mE (8838) AUDIO_ELEMENT: [i2s] Element already stopped␛[0m
␛[0;33mW (8848) AUDIO_PIPELINE: There are no listener registered␛[0m
␛[0;32mI (8848) AUDIO_PIPELINE: audio_pipeline_unlinked␛[0m
␛[0;33mW (8858) AUDIO_ELEMENT: [i2s] Element has not create when AUDIO_ELEMENT_TERMINATE␛[0m
␛[0;33mW (8868) AUDIO_ELEMENT: [mp3] Element has not create when AUDIO_ELEMENT_TERMINATE␛[0m

Now when I start the example and listen through the headphones, I hear a short klick every time the stream starts, followed by 7 seconds of silence and another, more quiet click when the stream terminates. In the console output you can see I added a register dump of all the register values after configuration. I checked them with the register tables in the data sheet and they are correct. So I started monitoring the bus with an LA (Screenshots added in the attachments) and here is what I found: In both, PCM Mode and I2s Mode (Philips Standard), the ESP seems to be a one bit short.

I2S_16Bit_Philips.png (18.91 KiB) Viewed 4402 times

I2S_16Bit_PCM.png (43.04 KiB) Viewed 4402 times

Unfortunately, I don't have a LyraT-Board here to reproduce this scenario, however I didn't touch the I2S driver in any way. Of course I checked back with the I2S timing diagrams in the ESP32 reference manual and it really seems to confirm the issue. Here is the Philips-Standard diagram with the data starting on the second rising edge after the Word-Clock Falling Edge:

ESP_RefMan_Philips_MODE.png (26.92 KiB) Viewed 4402 times

Do I miss something here or is there actually something rotten in the state of Denmark?