ESP32C3 USB CDC loosing characters on reception of stream of data.

[Danjovic]

Note to moderators: If this section of the forum is still not the right place to post please point me to the best place to do it.

Couldn't track the exact origin of the problem related below and I haven't found exactly the same problem on any of the forums and bug tracks I have looked for, although I have tried a lot of insights from similar problems. After days of struggle and failures I have stripped down the code to a minimum and yet I have the same problem.

I am using the ESP32C3 Supermini on an application that requires the reception and processing of about 100kBytes of ASCII data (intel hex). The board is connected to a host computer using the native USB CDC interface. The data is transferred by pasting the data on a serial terminal application on PC but the board keeps loosing characters on reception.

EDIT: The same behavior happens with the ESP32S2 and xiao ESP32C6 boards.

More details below.

Board: ESP32C3-Supermini
Hardware Setup: Windows PC(Win 11), USB-C cable, ESP32-C3 Supermini
Development Platform: Arduino 2.3.3, ESP32 board package 3.1.0
Application: Eprom emulator. It requires the reception and processing of about 100kBytes of ASCII data (intel hex file for a 32k eprom)

Issues found:
- Loosing characters on reception, even setting bigger (1k) buffers.
- No matter the baudrate, it 'eats' all the characters instantaneously. (for comparison, the same program running on an Arduino Nano at 9600bps took 3.8 seconds to receive the same amount of characters that the ESP32 got instantaneously)

Steps to reproduce:
1) Upload the code below

Code: Select all

void setup() {
  Serial.begin(9600);
}

void loop() {
  if (Serial.available()) {
    Serial.write(Serial.read());
  }
}

2) create a file with 110 lines like

Code: Select all

:100130003F0156702B5E712B722B732146013421C7
.... more 108 lines like the above 
:100130003F0156702B5E712B722B732146013421C7
:10010000214601360121470136007EFE09D2190140
:00000001FF

3) open the serial connection using PUTTY (or any other serial terminal program)

4) paste the file created in step 2.


Question:
Is there any workaround for this problem or is it inherent to the CDC peripheral on the ESP32C3 ?

Thanks!

[Sprite]

- Yep, the USB CDC ignores the baud rate setting and always works as fast as it can. That's because there is no physical UART involved; setting the baud rate is supported because a lot of programs expect that to work, but the setting is ignored.
- Not sure why you're losing characters... I don't expect it to, but does upgrading to the latest 3.2.0 arduino-esp32 release change things?

[Danjovic]

I have migrated to 3.2.0 and unfortunately the board still loose characters just the same.

I have also tested two more boards that I have here: xiao ESP32C6 and a red unbranded ESP32S2 and both loose characters too. The main difference is that the S2 to loose less characters (maybe due to CPU speed ?).

One thing that I have noticed is that the characters lost are not the same every time, for instance, sometimes I can see the last characters of the sequence pasted, sometimes I don't.

[Sprite]

I have also tested two more boards that I have here: xiao ESP32C6 and a red unbranded ESP32S2 and both loose characters too. The main difference is that the S2 to loose less characters (maybe due to CPU speed ?).

That is an interesting data point... the S2 and the C6 use entirely different hardware and software stacks to handle CDC-ACM; the fact that you see the issue with both seems to indicate the underlying ESP-IDF code is fine (or happens to be buggy in a way where they show the same symptomps, which is possible but less likely). That would point at either the upper Arduino layer or your software to have an issue.

[Danjovic]

I would bet my chips in the Arduino layer because the problem happens with the simple 'serial echo' code you see in the first post.

I should do some tests on the IDE then.

[Danjovic]

Ok, installed and configured ESPIDF 5.4.1

1) Compiled the example "peripherals\usb_serial_jtag\usb_serial_jtag_echo" for the 3 platforms (configured it first with set-target.
2) Configured Putty for the respective serial port and marked settings->terminal->implicit LF in ervery CR.
3) Paste the Ascii file (8294 characters, 189 lines)

Results:
ESP32C3 - Compile Ok, loose characters,

ESP32C6 - Compile Ok, loose characters,

ESP32S2 - Compile error on step 1001.
[1001/1003] Linking CXX executable usb_serial_jtag_echo.elf
FAILED: usb_serial_jtag_echo.elf

[Sprite]

1) Compiled the example "peripherals\usb_serial_jtag\usb_serial_jtag_echo" for the 3 platforms (configured it first with set-target.
2) Configured Putty for the respective serial port and marked settings->terminal->implicit LF in ervery CR.
3) Paste the Ascii file (8294 characters, 189 lines)

Can you link/upload/... that ascii file here so I can see if I can replicate it?

EDIT: Hm. It looks like the issue in that particular example is that the line that starts with ESP_LOG_BUFFER_HEXDUMP takes a long time as it tries to write to the standard UART; removing that fixes the issue for me. That still should not happen, however; not reading the UART should apply backpressure and not drop bytes on the ground. Let's see if I can find the root issue...

Edit2: That is unfortunate: it seems that the USB-jtag-serial driver simply drops data on the ground if it cannot fit it into the ring buffer anymore. A hacky solution is to increase the RX ring buffer size by a lot so you can buffer everything you need, but that's obviously not a proper solution. I'll put in a ticket to fix this in our Jira.

[Danjovic]

I'll put in a ticket to fix this in our Jira.

Thank you Sprite.

I believe that the ASCII file is not necessary anymore, right ? It is just a plain ascii .hex file representing around 2900 bytes of data. I tried to attach it on the page but I got an 'http error' message.

[Sprite]

I'll put in a ticket to fix this in our Jira.

Thank you Sprite.

I believe that the ASCII file is not necessary anymore, right ? It is just a plain ascii .hex file representing around 2900 bytes of data. I tried to attach it on the page but I got an 'http error' message.

You're correct; I can replicate it with an intel hex file with just random data in it.

[Danjovic]

Hello Sprite,

Any news about this issue? Can you point me a link where I can keep track on this issue without bothering you and the other users of this forum ?

Thanks!