I don't have this ESP32 to test with but many times with dual processors audio sometimes gets distorted due to switching between processors.

Sinewave test is forced to one processor, that is what I understand. I could of gotten the wrong information on this.

Try setting the audio to one processor ...