solution: use MALLOC_CAP_DMA for buffers

[bbulkow]

Hello,

I chased a problem with ESP-ADF for weeks, and found a solution (sort of).

I created a simple pipeline reading from a WAV file on SD card, and writing to the output DAC. My hardware is the AIThinker Audio-Kit hardware (I know, not a loved board), one made by ECBuying.

It mostly worked the first time, which was exciting, but I had a problem where every few minutes (from 1.5 minutes to 6 minutes), there was a bad glitch in the audio - always the same kind, 6 fast holes of 512ms and one of 300ms. Then it would play perfectly for minutes again.

After literally several months, I have tracked down the solution, SORT OF.

What I find is the read() call must be called with a MALLOC_CAP_DMA buffer. If it is just called with MALLOC_CAP_INTERNAL, the buffer is allocated out of IRAM not DRAM, on this platform. IRAM is not 8BIT nor DMA. This causes code in the SDMMC driver to - for every sector - call malloc to get a one-sector DMA-able buffer, call the SDMMC driver to read the data, then memcpy it to the desired buffer, then free. Yes, dynamically. There is no attempt to make the buffer static. This pattern also happens if the read() buffer is SPIRAM, and the glitches happen too, implying the problem is not with either IRAM or SPIRAM, but with malloc() generally.

If you make the buffer MALLOC_CAP_DMA to begin with, you avoid the malloc(), memcpy, free().

What I can't figure out is where in ESP-ADF to make the change. I had to write all my own code in raw ESP-IDF, port over the ES8388 drivers, write my own ringbuf and header parsers, etc, to get audio playing, then to reproduce the glitch, then to find the solution.

I don't know why the extra malloc() / memcpy / free would cause problems so irregularly. When the glitch is happening, it's not CPU bound, the read call is sleeping, I would assume on the malloc() or free() (in both code paths the SDMMC call is to dram data). When the code *isn't* glitching, CPU use is low (under 5% of the app core, nothing on the system core). I have disabled wifi and removed the tasks, same for bluetooth, as we often point to them as suspects. This points to a possible issue in malloc() or free() itself, and it's being called at very different frequency, which is certainly complex code, or an issue with the board I'm using - but I can't think of what hardware problem would cause this. I looked at power supply issues and heat issues and found nothing. No brownout detection or watchdog is being triggered - just a very very slow group of calls.

It's certainly a really good idea to avoid this much malloc() and free()! Every sector is a lot!!!! Even if others don't have the glitch, the performance impact could be very large.

In any case, I would hope someone at Espressif who maintains ESP-ADF could track down where this buffer is allocated for a pipeline that originates in FAT, and either put it in DRAM, or tie it to a menuconfig so that the allocation pool can be configured. In my own code, I am also using Static allocated ringbufs so I can be sure to put the ringbuf data in DRAM so when I call i2s_send it works as expected.

It took a very long time to find the resolution to this issue. I hope you can reproduce it and resolve it.

[bbulkow]

The solution I have found, with ESP-ADF 2.7, is the "audio_calloc" call in audio_element.c at line 469 near the top of audio_element_task. Changing this to a heap_caps_malloc with MALLOC_CAP_DMA seems the lowest impact change I can do and still get non-glitching audio.

It will still use SPIRAM for other elements of the audio system. The abstraction in fatfs_stream.c makes it hard to figure out how to allocate the buffer just in the case of fatfs_stream, greater surgery to ESP-ADF would be required to create both SPIRAM and DMA type audio elements, and override the type only in the case of fatfs_stream.

Maybe this helps someone.