Low-latency (<7ms) audio streaming with ESP-NOW

[map_93]

Hello everyone,
I am new to the ESP32 world. I am trying to develop a battery operated audio wireless transmission system for musical instruments.
The basic requirements are:
Sample rate: 48kHz
Resolution: 16bit or 24bit
Mono/Stereo: Stereo

I was planning on using ESP-NOW for communication between two EPS32s. My initial tests with ESP-NOW have been underwhelming though. With a basic Arduino sketch example, sending 250 bytes with acknowledge requires 4-5ms on average, with peaks of 10-12ms. I was looking into ESP-IDF for more customisation/better performance, but I got lost in the number of options and settings.
Can anyone confirm if what I am trying to achieve is even possible? Is it normal for sending times to vary so much?
As mentioned, I would like to keep latency low (ideally less than 7ms) as the system is meant for live in-ear monitoring.
Thank you very much for any help/advice!