ESP32S3 with OV3660 camera Autonomous Robot

[pawson]

Hello,
I am not sure how to realize my project and it would be awesome to get some tips about choosing the right path. The main hero of this project is an autonomous mobile robot which will detect object (simple thresholding I guess), picks it up and returns to a base. I would love your guidance on how to divide responsibilities between components.

1) Solo usage of ESP32
Modify the CameraWebServer sketch and implement appropriate color thresholding plus basic morphological operations on the board. The ESP32 also handles motor control.

2) ESP32 as a streaming node + laptop
Use the Seeed Xiao ESP32-S3 Sense with OV3660 on the robot to capture video telemetry and stream and them over wifi. A laptop (eg. Python/OpenCV) process the frames and sends motion commands back to the robot over wifi.

3) ESP32 + Raspberry Pi on board
The ESP32 is responsible for low lvl motor control (PWM, drivers, encoders, sensors), while the Raspberry Pi performs image processing and acts as a high lvl control unit.

Which approach would you recommend given my goals?
Thank you a lot
Pawel J

[ahsrabrifat]

I think you can initially start with ESP32 streaming to laptop. This lets you rapidly prototype vision algorithms in Python/OpenCV and confirm that your detection pipeline works reliably. You’ll learn quickly what kind of compute you really need.