LWIP Stack Behavior - I must be doing something wrong

Post Reply
jhinkle
Posts: 35
Joined: Wed Aug 29, 2018 3:17 pm

LWIP Stack Behavior - I must be doing something wrong

Postby jhinkle » Wed Jan 07, 2026 11:17 pm

I've been using the FreeRTOS TCP stack for years now with ARM MCUs.

I have a project that uses a ESP32 and I'm using the IDF with Vscode.

I use a Socket and set the socket Opt of TCP_NODELAY to true so data is transmitted without delay.

I'm using TCP to communicate with a PC program so I have visibility to all send and receive operations.

Sockets on the PC and in the FreeRTOS stack queue send messages so the application and send multiple packets to the stack without delay.

What I have found with the LWIP stack (Sockets) is if a send/write to the socket occurs with little to no time between them, the Second packet NEVER goes out on the wire. I've confirmed this by trapping received packets on the PC and capturing activity via wireshark.

ONLY after I introduced a DELAY (I'm using 6 seconds) between writes then all packets are sent. I know 6 secs is very long and something much smaller could be used -- BUT --- I found there must be a Delay between writes for the stack to perform.

I've implemented a SEND Queue and state machine to manage my socket writes so it's working fine now but I would like to know why the LWIP stack requires the delay.

Thanks in advance for any comments.
Top
nopnop2002
Posts: 347
Joined: Thu Oct 03, 2019 10:52 pm
Contact:
Contact nopnop2002

Re: LWIP Stack Behavior - I must be doing something wrong

Postby nopnop2002 » Thu Jan 08, 2026 5:23 am

The TCP_NODELAY option works fine.

Code: Select all

/*
 * SPDX-FileCopyrightText: 2022 Espressif Systems (Shanghai) CO LTD
 *
 * SPDX-License-Identifier: Unlicense OR CC0-1.0
 */
#include "nvs_flash.h"
#include "esp_netif.h"
#include "protocol_examples_common.h"
#include "esp_event.h"

#include <sys/socket.h>

int tcp_client(int option)
{
        printf("CONFIG_EXAMPLE_SERVER_PORT=%d\n", CONFIG_EXAMPLE_SERVER_PORT);
        printf("CONFIG_EXAMPLE_SERVER_ADDR=%s\n", CONFIG_EXAMPLE_SERVER_ADDR);

        struct sockaddr_in server;
        int sock, i;
        ssize_t n;
        char buf[1]="9";

        sock = socket(AF_INET, SOCK_STREAM, 0);

        server.sin_family = AF_INET;
        server.sin_port = htons(CONFIG_EXAMPLE_SERVER_PORT);
        server.sin_addr.s_addr = inet_addr(CONFIG_EXAMPLE_SERVER_ADDR);

        setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, &option, sizeof(option));
        connect(sock, (struct sockaddr *)&server, sizeof(server));

        for(i=0; i<100; i++) {
                n = write(sock, buf, sizeof(buf));
                if(n == -1) {
                        perror("write");
                        return 1;
                }
        }
        close(sock);
        return 0;
}

void app_main(void)
{
        ESP_ERROR_CHECK(nvs_flash_init());
        ESP_ERROR_CHECK(esp_netif_init());
        ESP_ERROR_CHECK(esp_event_loop_create_default());

        /* This helper function configures Wi-Fi or Ethernet, as selected in menuconfig.
         * Read "Establishing Wi-Fi or Ethernet Connection" section in
         * examples/protocols/README.md for more information about this function.
         */
        ESP_ERROR_CHECK(example_connect());

        tcp_client(1);
}

All packets are 1 byte in size.
No packet loss.

Code: Select all

$ sudo tcpdump -i enp2s0 dst port 11111 -nn
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on enp2s0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:26:24.462971 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [S], seq 727195953, win 5760, options [mss 1440], length 0
14:26:24.481543 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [.], ack 2307707678, win 5760, length 0
14:26:24.483152 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 0:1, ack 1, win 5760, length 1
14:26:24.484178 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 1:2, ack 1, win 5760, length 1
14:26:24.486915 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 2:3, ack 1, win 5760, length 1
14:26:24.487203 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 3:4, ack 1, win 5760, length 1
14:26:24.487203 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 4:5, ack 1, win 5760, length 1
14:26:24.494690 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 5:6, ack 1, win 5760, length 1
14:26:24.494691 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 6:7, ack 1, win 5760, length 1
14:26:24.494691 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 7:8, ack 1, win 5760, length 1
14:26:24.496242 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 8:9, ack 1, win 5760, length 1
14:26:24.498892 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 9:10, ack 1, win 5760, length 1
14:26:24.500050 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 10:11, ack 1, win 5760, length 1
14:26:24.502974 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 11:12, ack 1, win 5760, length 1
14:26:24.504307 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 12:13, ack 1, win 5760, length 1
14:26:24.506878 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 13:14, ack 1, win 5760, length 1
14:26:24.508054 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 14:15, ack 1, win 5760, length 1
14:26:24.510580 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 15:16, ack 1, win 5760, length 1
14:26:24.521370 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 16:17, ack 1, win 5760, length 1
14:26:24.521371 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 17:18, ack 1, win 5760, length 1
14:26:24.521371 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 18:19, ack 1, win 5760, length 1
14:26:24.521372 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 19:20, ack 1, win 5760, length 1
14:26:24.521372 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 20:21, ack 1, win 5760, length 1
14:26:24.521372 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 21:22, ack 1, win 5760, length 1
14:26:24.521372 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 22:23, ack 1, win 5760, length 1
14:26:24.521373 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 23:24, ack 1, win 5760, length 1
14:26:24.530014 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 24:25, ack 1, win 5760, length 1
14:26:24.530015 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 25:26, ack 1, win 5760, length 1
14:26:24.530015 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 26:27, ack 1, win 5760, length 1
14:26:24.530015 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 27:28, ack 1, win 5760, length 1
14:26:24.532199 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 28:29, ack 1, win 5760, length 1
14:26:24.534108 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 29:30, ack 1, win 5760, length 1
14:26:24.534108 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 30:31, ack 1, win 5760, length 1
14:26:24.534109 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 31:32, ack 1, win 5760, length 1
14:26:24.540077 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 32:33, ack 1, win 5760, length 1
14:26:24.540540 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 33:34, ack 1, win 5760, length 1
14:26:24.540541 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 34:35, ack 1, win 5760, length 1
14:26:24.540541 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 35:36, ack 1, win 5760, length 1
14:26:24.541664 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 36:37, ack 1, win 5760, length 1
14:26:24.545053 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 37:38, ack 1, win 5760, length 1
14:26:24.546355 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 38:39, ack 1, win 5760, length 1
14:26:24.560389 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 39:40, ack 1, win 5760, length 1
14:26:24.561027 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 40:41, ack 1, win 5760, length 1
14:26:24.561027 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 41:42, ack 1, win 5760, length 1
14:26:24.561027 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 42:43, ack 1, win 5760, length 1
14:26:24.561028 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 43:44, ack 1, win 5760, length 1
14:26:24.561028 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 44:45, ack 1, win 5760, length 1
14:26:24.561028 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 45:46, ack 1, win 5760, length 1
14:26:24.561028 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 46:47, ack 1, win 5760, length 1
14:26:24.561029 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 47:48, ack 1, win 5760, length 1
14:26:24.561055 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 48:49, ack 1, win 5760, length 1
14:26:24.561056 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 49:50, ack 1, win 5760, length 1
14:26:24.561056 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 50:51, ack 1, win 5760, length 1
14:26:24.561056 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 51:52, ack 1, win 5760, length 1
14:26:24.561056 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 52:53, ack 1, win 5760, length 1
14:26:24.561057 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 53:54, ack 1, win 5760, length 1
14:26:24.562852 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 54:55, ack 1, win 5760, length 1
14:26:24.564339 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 55:56, ack 1, win 5760, length 1
14:26:24.565778 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 56:57, ack 1, win 5760, length 1
14:26:24.568246 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 57:58, ack 1, win 5760, length 1
14:26:24.574814 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 58:59, ack 1, win 5760, length 1
14:26:24.575439 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 59:60, ack 1, win 5760, length 1
14:26:24.575440 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 60:61, ack 1, win 5760, length 1
14:26:24.575440 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 61:62, ack 1, win 5760, length 1
14:26:24.575440 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 62:63, ack 1, win 5760, length 1
14:26:24.575441 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 63:64, ack 1, win 5760, length 1
14:26:24.576758 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 64:65, ack 1, win 5760, length 1
14:26:24.579099 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 65:66, ack 1, win 5760, length 1
14:26:24.585011 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 66:67, ack 1, win 5760, length 1
14:26:24.585540 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 67:68, ack 1, win 5760, length 1
14:26:24.585540 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 68:69, ack 1, win 5760, length 1
14:26:24.585540 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 69:70, ack 1, win 5760, length 1
14:26:24.585541 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 70:71, ack 1, win 5760, length 1
14:26:24.585541 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 71:72, ack 1, win 5760, length 1
14:26:24.589395 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 72:73, ack 1, win 5760, length 1
14:26:24.589763 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 73:74, ack 1, win 5760, length 1
14:26:24.589764 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 74:75, ack 1, win 5760, length 1
14:26:24.589764 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 75:76, ack 1, win 5760, length 1
14:26:24.593133 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 76:77, ack 1, win 5760, length 1
14:26:24.596844 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 77:78, ack 1, win 5760, length 1
14:26:24.596845 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 78:79, ack 1, win 5760, length 1
14:26:24.596846 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 79:80, ack 1, win 5760, length 1
14:26:24.598471 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 80:81, ack 1, win 5760, length 1
14:26:24.600943 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 81:82, ack 1, win 5760, length 1
14:26:24.603331 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 82:83, ack 1, win 5760, length 1
14:26:24.604942 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 83:84, ack 1, win 5760, length 1
14:26:24.607578 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 84:85, ack 1, win 5760, length 1
14:26:24.610115 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 85:86, ack 1, win 5760, length 1
14:26:24.612026 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 86:87, ack 1, win 5760, length 1
14:26:24.613786 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 87:88, ack 1, win 5760, length 1
14:26:24.616926 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 88:89, ack 1, win 5760, length 1
14:26:24.618182 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 89:90, ack 1, win 5760, length 1
14:26:24.619832 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 90:91, ack 1, win 5760, length 1
14:26:24.622320 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 91:92, ack 1, win 5760, length 1
14:26:24.623498 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 92:93, ack 1, win 5760, length 1
14:26:24.624776 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 93:94, ack 1, win 5760, length 1
14:26:24.627951 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 94:95, ack 1, win 5760, length 1
14:26:24.630123 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 95:96, ack 1, win 5760, length 1
14:26:24.642847 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 96:97, ack 1, win 5760, length 1
14:26:24.642848 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 97:98, ack 1, win 5760, length 1
14:26:24.642848 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 98:99, ack 1, win 5760, length 1
14:26:24.642848 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [P.], seq 99:100, ack 1, win 5760, length 1
14:26:24.651378 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [F.], seq 100, ack 1, win 5760, length 0
14:26:24.662775 IP 192.168.0.158.56466 > 192.168.0.46.11111: Flags [.], ack 2, win 5759, length 0
Top
Post Reply

Return to “ESP-IDF”

Who is online

Users browsing this forum: Bing [Bot], ChatGPT-User, meta-externalagent, YisouSpider and 6 guests