Parallel IO: ESP32-C5 image streaming with DVP Cameras (ESP32-C5-CAM)

[HaqqScripter]

I have successfully be able to interface DVP camera with ESP32-C5 (Or targets supporting Parallel IO peripheral) to achieve 5GHz connectivity without the need to use other SoCs.
This is useful for example, a cost-effective single-chip applications requiring Wi-Fi 6 connectivity. For my use case, it is a dual-band Wi-Fi surveillance camera.
For OV5640, I was able to get ~28fps up to SVGA, ~32fps at HVGA, ~22fps on 1280x960, ~30fps on HD, ~13.5 (max) on FHD/WUXGA as well as QXGA/QSXGA (~10fps max) in its default setting.
You can refer to High Performance Mode (HPM) to require higher frame rates on OV5640 and OV3660. I would say it is comparable to Xtensa chips.

Overview of the connection of the ESP32-C5 & OV5640 camera.

video_transferring_esp32_c5.gif (15.53 MiB) Viewed 17125 times

The image streaming test.

video_transferring_esp32_c5_pt2.gif (14.03 MiB) Viewed 17125 times

For convenience, I have also made an ESP-IDF component esp_cam_io_parl v0.1.0-beta.10:
ESP Component Registry: https://components.espressif.com/compon ... am_io_parl
GitHub: https://github.com/haqqscripter/esp_cam_io_parl


Introducing High Performance Mode (HPM):
OV5640 (esp_cam_io_parl v0.1.0-beta.7):

• Allow images to be captured around 15FPS at maximum resolution (2592x1944), you can also achieve 20FPS QHD/FHD with this setting enabled.

OV3660 (esp_cam_io_parl v0.1.0-beta.10):

• Allow images to be captured around 20FPS at maximum resolution (2048x1536), you can also achieve 30FPS FHD with this setting enabled.

A stable 24MHz XCLK is preferred. View README.md on esp_cam_io_parl for more details.


ESP-IDF component features checklist before official version release:

• Rewrite esp_cam_io_parl headers and C code (Using esp_driver_parlio currently has some limitations). [Not Complete]

• Merge and rewrite esp_camera_sensor into esp_cam_io_parl. [Pending]

• ETM support for some DVP signals (trigger PARLIO RX start). [Not Complete]

• Allow methods to replicate the valid signal for targets that doesn't have sufficient bit width. [Not Complete]

You can now produce your own camera development board based on the ESP32-C5:
ESP32-C5-CAM Hardware: https://github.com/HaqqScripter/ESP32-C5-CAM_Hardware

[encryptior]

Hey, we are looking to connect our ov5640 camera to esp32 c5 dev board. I dont have much hardware experience. If you could help me out on how you managed to connect them wopuld be of great help! Thanks

[HaqqScripter]

Hey, we are looking to connect our ov5640 camera to esp32 c5 dev board. I dont have much hardware experience. If you could help me out on how you managed to connect them wopuld be of great help! Thanks

Hello, thanks for reaching out! Well, since the ESP32-C series, including ESP32-C5 have limited number of GPIOs (In this case: 22), I tend to buy OV5640 camera modules with integrated crystal (most of them are 24MHz), so you would occupy at least 11 GPIO pins (8 data + 1 pixel clock + 2 SCCB pins) on the ESP32-C5. Connecting them is simple enough, hardware side. For prototypes purposes, I just use a breadboard and a few short jumper wires (5cm works best), connecting the wires from the camera pins to the ESP32-C5-DevkitC-1, which you set it in your application code. I did attempt to use this with PCBs so it would have less interference to sample higher PCLK rate.

For building applications, you can refer to the ESP-IDF (esp_cam_io_parl) component examples and try it out on your ESP32-C5-DevkitC-1. I've used the Parallel IO peripheral on ESP32 compatible targets, each have their limitations, such as HREF signals can't be used on a 8-bit DVP on ESP32-C5.

In my case, the connection from the OV5640 module to ESP32-C5-DevkitC-1 is this:
SDA -> 26, SCL -> 25, PCLK -> 2, D0 -> 10, D1 -> 9, D2 -> 8, D3 -> 7, D4 -> 6, D5 -> 1, D6 -> 0, D7 -> 3

[encryptior]

Hey, I was trying to connect the camera and esp32c5 devkit. But I have a fundamental issue. I have an OV5640 camera with a ribbon cable(used in M5Stack esp32TimerCAM) and I got this camera breakout: https://roboticsdna.in/product/ffc-fpc- ... ykw-NwEP4w to experiment this camera with esp32c5 chip. I am unsure how do I get the pin config on the ribbon cable so I can know the pinout to use the camera. Do you have some inputs on how can I know this unknown.
I am attaching the image of camera module which is beign used in the M5TimerCam for your reference

[encryptior]

I actually was able to get the pin mapping and connected the relevant pins and tested the

Code: Select all

Simple Camera Capture

code and got successful image capture logs, though when I tried again after some time, I now am only getting capture failed due to timeout errors. Do you know what could be going wrong here?

[romanlh]

Thanks for providing the camera example.

Just as an additional note: there appears to be a bug in the SDK related to the sampling edge.

In the camera example, config->pclk_sample_edge is never initialized, so it effectively defaults to PARLIO_SAMPLE_EDGE_NEG (0). As a result, parlio_rx_soft_delimiter_config_t.sample_edge is also set to NEG, yet the code still works.

Code: Select all

        parlio_rx_soft_delimiter_config_t rx_delimiter_config = {
            .sample_edge = config->pclk_sample_edge,
            .eof_data_len = esp_cam_io_parl->payload_size,
            .timeout_ticks = 0,
        };

Code: Select all

  typedef enum {
    PARLIO_SAMPLE_EDGE_NEG, /*!< Sample data on falling edge of clock */
    PARLIO_SAMPLE_EDGE_POS, /*!< Sample data on rising edge of clock */
} parlio_sample_edge_t;

Here is the first HREF after VSYNC. The bytes ff f8 (low bits → f and 8) should definitely be sampled on the rising edge:

ffd8.png (38.19 KiB) Viewed 15626 times

In C6 driver code comments are broken: https://github.com/espressif/esp-idf/bl ... ruct.h#L72

Code: Select all

        /** rx_clk_edge_sel : R/W; bitpos: [26]; default: 0;
         *  Write 0 to enable sampling data on the rising edge of rx clock. Write 0 to enable
         *  sampling data on the falling edge of rx clock.
         */

I can not find register definition for esp32c5, but for esp32c6, O in PARL_IO_RX_LEVEL_SUBMODE_SEL means Rising Edge.

[HaqqScripter]

I actually was able to get the pin mapping and connected the relevant pins and tested the

Code: Select all

Simple Camera Capture

code and got successful image capture logs, though when I tried again after some time, I now am only getting capture failed due to timeout errors. Do you know what could be going wrong here?

I believe it can be due to power glitching, which may cause the OV5640 to accidentally reset though power cycle, and halts the DVP port as well, and based from the information that you had given to me, using the breakout board with jumper wires to connect to the ESP32-C5 are definitely the cause .

[HaqqScripter]

Thanks for providing the camera example.

Just as an additional note: there appears to be a bug in the SDK related to the sampling edge.

In the camera example, config->pclk_sample_edge is never initialized, so it effectively defaults to PARLIO_SAMPLE_EDGE_NEG (0). As a result, parlio_rx_soft_delimiter_config_t.sample_edge is also set to NEG, yet the code still works.

Code: Select all

        parlio_rx_soft_delimiter_config_t rx_delimiter_config = {
            .sample_edge = config->pclk_sample_edge,
            .eof_data_len = esp_cam_io_parl->payload_size,
            .timeout_ticks = 0,
        };

Code: Select all

  typedef enum {
    PARLIO_SAMPLE_EDGE_NEG, /*!< Sample data on falling edge of clock */
    PARLIO_SAMPLE_EDGE_POS, /*!< Sample data on rising edge of clock */
} parlio_sample_edge_t;

Here is the first HREF after VSYNC. The bytes ff f8 (low bits → f and 8) should definitely be sampled on the rising edge:
ffd8.png

In C6 driver code comments are broken: https://github.com/espressif/esp-idf/bl ... ruct.h#L72

Code: Select all

        /** rx_clk_edge_sel : R/W; bitpos: [26]; default: 0;
         *  Write 0 to enable sampling data on the rising edge of rx clock. Write 0 to enable
         *  sampling data on the falling edge of rx clock.
         */

I can not find register definition for esp32c5, but for esp32c6, O in PARL_IO_RX_LEVEL_SUBMODE_SEL means Rising Edge.

Yes, I'm glad that you've reached out to this. Earlier in development, I was investigating the issue on why it didn't sample correctly when I used the data sampling on the positive edge PARLIO_SAMPLE_EDGE_POS (I've tested on ESP32-C5/C6/H2) which then I've realized, the enum values may have been swapped/wrongly named, or maybe it was referring to the TX side of the Slave device which updates the data on defined edge (either falling or rising), I was unsure, but the naming seems to be directing to the PARLIO RX itself so I guess it could be wrongly labeled. The camera did latch the data lines on negative-edge and that is why I've exposed this property:

Code: Select all

.pclk_sample_edge 

... and let it be by default value. So users can set the sampling edge with their heart's content. I was also unsure whether the issue is only occurring on my end.
I will definitely address this in the esp_cam_io_parl ESP-IDF component as well, as the next beta revision was already undergoing for over two months .

[romanlh]

Does anyone know the correct way to calculate PCLK frequency and FPS from the OV5640 configuration parameters?

I am trying to verify whether the VCO, SYSCLK, and PCLK frequencies are within valid ranges when using a 24 MHz XCLK.

I am implementing a calculator:
https://romanlut.github.io/hx-esp32-cam ... lator.html

I followed the formulas used in the esp32-camera driver:
https://github.com/espressif/esp32-came ... 640.c#L200

However, the calculated values do not match the actual measured behavior.

[HaqqScripter]

Does anyone know the correct way to calculate PCLK frequency and FPS from the OV5640 configuration parameters?

I am trying to verify whether the VCO, SYSCLK, and PCLK frequencies are within valid ranges when using a 24 MHz XCLK.

I am implementing a calculator:
https://romanlut.github.io/hx-esp32-cam ... lator.html

I followed the formulas used in the esp32-camera driver:
https://github.com/espressif/esp32-came ... 640.c#L200

However, the calculated values do not match the actual measured behavior.

Unfortunately, I can't really figure out the exact PCLK frequency, assuming you've measured the pulses of the gated PCLK (not the average frequency), but still this is a good tool.