[esp_http_client]: how can I tell client state?

[Bryght-Richard]

I'm hoping to share an esp_http_client connection with `is_async = true` between a few different purposes. I need to ensure that any previous request has finished before changing the client settings to start a new HTTP request.

I can't seem to find a way to do so.

Here's what I think a generic request would look like:

Code: Select all

  esp_http_client_handle_t client;.
  
  void setup_client(void){
    //Somewhere Once
    esp_http_client_config_t config = {
      .url = "...",
      .is_async = true,
      .event_handler = ...,
      .crt_bundle_attach = esp_crt_bundle_attach,
    };
    client = esp_http_client_init(&config);
  }

void do_example_request(void){
    //Wait for client to complete outstanding request
    while(!esp_http_client_idle(client) && ESP_ERR_HTTP_EAGAIN == esp_http_client_perform(client)){
        //esp_http_client_perform() will run the event handler
    }

    //These might be different from any previous request,
    //so we cannot set them until we know the client is idle(either disconnected, or between requests)
    esp_http_client_set_header(client, "...", "...");
    esp_http_client_set_method(client, HTTP_METHOD_GET);
    esp_http_client_open(client, 0);
}

void idle_poll(void){
    //Wait for client to complete outstanding request
    if(!esp_http_client_idle(client)){
        //esp_http_client_perform() will run the event handler
        esp_http_client_perform(client);
    }
}

I don't want to call `esp_http_client_perform()` unless I know it's needed, and the central point to know this would be the client itself rather than each possible request sender. Am I missing something obvious? Other approaches? Thank you for your consideration.

[MicroController]

You could use a FreeRTOS queue as a simple, thread-safe, wait-able pool of client handles. Take a handle from the queue, use it, put it back into the queue when done.

[Bryght-Richard]

You could use a FreeRTOS queue as a simple, thread-safe, wait-able pool of client handles. Take a handle from the queue, use it, put it back into the queue when done.

This is a great idea for other applications. The server(I don't control) temporarily bans clients for too many parallel requests, and serializing requests through a single client handle solves that nicely for me. And I have to send extra requests to keep the connection open.

The requests are usually fast too, like 30-200ms depending on internet routing, but the ESP32-S3 setting up a new connection is 1-2 seconds, so reuse is critical.

[MicroController]

If you think about it some more, you'll realize that the queue is a great idea for your application too.

Especially since you were even willing to consider

Code: Select all

while(!esp_http_client_idle(client)...

[Bryght-Richard]

If you think about it some more, you'll realize that the queue is a great idea for your application too.

Especially since you were even willing to consider

Code: Select all

while(!esp_http_client_idle(client)...

^^ I was a bit surprised to end up thinking this was a good solution too. In most cases, we have 0 or 1 outstanding + pending requests, and all are very small requests and responses. So that loop shouldn't be hit often. Testing shows its almost always far faster to wait for an outstanding request to finish than to reconnect another. It looks like keeping too many connections open with dummy requests counts against us at the server's rate-limiter.

[MicroController]

Hint: A "pool"/"queue" can have a size/capacity of 1

A FreeRTOS queue should be the easiest way to implement mutually-exclusive use of an object with the inherent ability to block while the object is not available.

You can of course also 'manually' associate a client handle with a mutex or semaphore to synchronize use of the handle between tasks.

[MicroController]

By the looks of it, you probably don't even need to loop/wait before sending the "keep-alive" request. You could just check, for instance every 5 seconds, if the client handle is 'available'; if it is, send a "keep-alive"; if not, do nothing and wait for the next 5 second interval to elapse.
Or you could re-start a timer (of e.g. 5s) each time a request completes. (Only) if no new request occurs within 5 seconds of the last one completing, the timer 'fires' and you send a "dummy"/"keep-alive" request.

Still need to ensure mutually-exclusive use of the client handle across tasks though.

[Bryght-Richard]

I've got my code structured such that each esp_http_client is only ever used by a single thread for its lifetime.

> Or you could re-start a timer (of e.g. 5s) each time a request completes. (Only) if no new request occurs within 5 seconds of the last one completing, the timer 'fires' and you send a "dummy"/"keep-alive" request.

This is how I'm planning to trigger sending keep-alive requests, but while a "keep-live" response is pending I still need to defer any user-triggered requests.

[MicroController]

esp_http_client connection with `is_async = true`

I've got my code structured such that each esp_http_client is only ever used by a single thread for its lifetime

while a "keep-live" response is pending I still need to defer any user-triggered requests.

Guess we're not getting anywhere here.

Ok, how about this:
1. Take the client handle from the pool/queue, blocking until it's available if needed.
2. Set up the client instance for the request, call perform().
3. In the callback, handle the events as usual. When you receive an HTTP_EVENT_ON_FINISH, put the client handle back into the pool/queue.

[Bryght-Richard]

> Guess we're not getting anywhere here.

I'm not sure what you mean, but I found your feedback valuable. At least to confirm the other thoughts that led me here.

Setting is_async creates no additional threads, it just allows the caller to poll esp_http_client_perform() as needed to trigger the event handlers. For some of my requests, I'd like to be able to send the request and not block the requesting thread. Keep-live requests are a great example of this - I don't really care about the response, but need to ensure it's been received before sending another request.