Missing 11 bytes from TCP download when ESP_AUTO_RECV set

[chrismerck]

I know this is ancient history at this point, but we are narrowing down a bug in our OTAU which often* results in 11 bytes missing from a TCP stream. That is, we `read` from the socket, and the total number of bytes that we are able to read is 11 less than is seen in Wireshark. Somewhere in lwIP those 11 bytes are being dropped.

(*) This used to happen only seldom, or for users with high-latency internet connections. Then it has occurred more frequently to the point that units are nearly un-upgradable. Thankfully we can also update from the HTTP server embedded in our app.

I found that disabling ESP_AUTO_RECV makes the problem go away. I suspect some kind of thread safety issue.

The question: Can anyone shed more light on this "ESP_AUTO_RECV" function? Why was it added? Was this an optimization? What can be expected when we disable it?

We are on ESP-IDF v3.3.4.

Commit that introduced ESP_AUTO_RECV: https://github.com/espressif/esp-lwip/c ... 4ffd7dc2a8

[chrismerck]

It turns out that the bug was in my code for handling the return value of `read()`.

I was doing:

Code: Select all

r = read(...);
if (r == EAGAIN || r == EWOULDBLOCK) return 0; // treat as timeout

which would treat receiving 11 (EAGAIN) bytes or 35 (EWOULDBLOCK) as a read timeout.

This bug was silently fixed 3 weeks ago in our code by o1 and explained to us today by Claude 3.7 Sonnet Thinking.