ESP32-S3 Korvo-2 Male-to-Female Voice Conversion via API (ESP-IDF + ESP-ADF)

[vipulgupta]

# ESP32-S3 Korvo-2 Male-to-Female Voice Conversion via API (ESP-IDF + ESP-ADF)

## 1. Project Overview

Hello everyone,

I am currently working on a "voice changer project (Male to Female voice conversion)" using the "ESP32-S3 Korvo-2 board". The goal of this project is to capture audio from the microphone on the Korvo-2 board, send it to a "voice conversion API", and then play the converted "female voice audio" through the speaker connected to the board.

The main idea is to offload the "voice conversion processing to a cloud-based model/API", since real-time voice conversion is computationally intensive for the ESP32.



## 2. Hardware

ESP32-S3 Korvo-2 development board
Onboard microphone array
Speaker connected to the I2S audio codec


## 3. Software Environment

ESP-IDF version : v5.5.2
ESP-ADF version : v2.7
Using the ESP-ADF audio pipeline for capturing and playing audio
HTTP communication to send audio data to the external API



## 4. System Workflow

The intended workflow of the system is:

Microphone (Korvo-2)
↓
ESP32 Audio Pipeline (ESP-ADF)
↓
Capture audio buffer
↓
Send audio via HTTP request to Voice Conversion API
↓
API performs Male → Female voice conversion
↓
Converted audio returned to ESP32
↓
ESP32 receives audio stream
↓
Playback through speaker



## 5. Current Challenges

While implementing this architecture, I am facing several issues:

1. Many voice conversion APIs have "very limited free credits".
2. Some APIs are designed for "file upload rather than real-time streaming".
3. Integrating **audio streaming between ESP32 and the API** is challenging.
4. Some services require "Python-based intermediate servers", which I would prefer to avoid if possible.



## 6. Questions

I would really appreciate guidance on the following:

1. Are there any "voice conversion APIs suitable for real-time or near real-time audio streaming"?
2. Has anyone implemented "audio streaming from ESP32 to a cloud AI model and received processed audio back"?
3. Are there any "recommended architectures for implementing voice conversion with ESP32"?


## 7. Additional Information

If anyone has experience with this type of implementation, please guide me. Your response would be very valuable for me.

Any suggestions or guidance would be greatly appreciated.

Thank you.

[Sprite]

What's your use case? Something you speak into, then 'live' converts the voice to female? If so, I'd take a long hard look at how much latency you can tolerate. Obviously, tens of seconds of latency is a problem all by itself, but even hundreds of milliseconds can be an issue: aside from the fact that lip sync may be 'off', it's really confusing to hear yourself back with a small delay.

[vipulgupta]

Thanks for your response.

At the moment, I don't have a specific real-time use case. This is mainly a project where I am experimenting with voice conversion using the ESP32-S3 Korvo-2 board.

For now, I am not focusing on real-time conversion. The workflow I am trying to implement is simpler:

1. I speak something into the microphone on the Korvo-2 board.
2. The ESP32 records the audio.
3. The recorded audio is sent to a voice conversion API using an HTTP request.
4. The API performs the male-to-female voice conversion.
5. The converted audio is returned from the API.
6. The ESP32 saves the converted audio to the SD card.
7. Finally, the audio is played through the speaker.

So latency is not a major concern for this stage of the project. I am mainly trying to understand the best way to integrate the ESP32 audio pipeline with a voice conversion API.

If you have any suggestions regarding APIs or system architecture for this type of workflow, I would really appreciate it.