File download with WifiClientSecure. I'm missing smtg...

bobo1974
Posts: 26
Joined: Fri Feb 08, 2019 2:14 pm

File download with WifiClientSecure. I'm missing smtg...

Postby bobo1974 » Thu Sep 16, 2021 9:14 am

I have build a sketch to download a file over https based on WifiClientSecure.
The purpose is not to use that file but rather to check the speed of the network.
Compiles fine, seems to work fine... Principle is that it will connect, check "client.available()", read from the buffer.

2 things I don't understand, after trying so many different things:

1 - what client.available() returns is always 16135. It does not matter if I am requesting a 1Mb or 10Mb file. So it looks I am missing smtg. It is supposed to be the number of "bytes", but it doesn't add up to me... It takes the same time no matter the file size!
What am I missing?

2 - in the while loop, when reading incoming data, If I serial.print the read data, it prints a lot of data so it seems I am indeed downloading. But if I don't use this data nore print anything in the loop, the transfer seems immediate. Even with 1Gb file....
Do I need to do something with the data? I started to think that If I am not, then it might actually not download it....


Code: Select all

#include <WiFiClientSecure.h>
#define buf_size 500
const char* ssid     = "XXXXXXXX";     // your network SSID (name of wifi network)
const char* password = "XXXXXXXX"; // your network password
int incoming = 1;
size_t readB;
size_t count = 0;
WiFiClientSecure client;
uint8_t inputBuffer[buf_size];

const char*  server = "proof.ovh.net";  // Server URL
const char* root_ca = \
                      "-----BEGIN CERTIFICATE-----\n" \
                      xxxx
                      "-----END CERTIFICATE-----\n";

void setup() {
  Serial.begin(115200);
  Serial.print("Attempting to connect to SSID: "); Serial.println(ssid);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(1000);
  }

  Serial.print("Connected to ");
  Serial.println(ssid);

  client.setCACert(root_ca);
  Serial.println("\nStarting connection to server...");
  if (!client.connect(server, 443))
    Serial.println("Connection failed!");
  else {
    Serial.println("Connected to server!");
    client.println("GET https://proof.ovh.net/files/10Mb.dat HTTP/1.1");
    client.println("Host: proof.ovh.net");
    client.println("Connection: close");
    client.println();

    while (client.connected()) {
      String line = client.readStringUntil('\n');
      Serial.println(line);
      if (line == "\r") {
        Serial.println("headers received");
        break;
      }
    }

    while (incoming) {
      incoming = client.available();
      readB = client.read(inputBuffer, incoming > buf_size ? buf_size : incoming);
      Serial.println(count);
      count = count + readB;
      delay(10);
    }
    client.stop();
    Serial.print("Finished: read bytes: "); Serial.println(count);
  }
}

void loop() {
}
Result for 10MB file : https://proof.ovh.net/files/10Mb.dat

Code: Select all

11:06:36.415 -> Attempting to connect to SSID: XXXXXXXX
11:06:36.461 -> [    52][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 0 - WIFI_READY
11:06:36.553 -> [   138][V][WiFiGeneric.cpp:272] _arduino_event_cb(): STA Started
11:06:36.553 -> [   140][V][WiFiGeneric.cpp:96] set_esp_interface_ip(): Configuring Station static IP: 0.0.0.0, MASK: 0.0.0.0, GW: 0.0.0.0
11:06:36.553 -> [   140][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 2 - STA_START
11:06:36.553 -> ...[  2279][V][WiFiGeneric.cpp:284] _arduino_event_cb(): STA Connected: SSID: SFR-2598, BSSID: 68:ff:7b:c7:82:62, Channel: 3, Auth: WPA2_PSK
11:06:38.669 -> [  2280][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 4 - STA_CONNECTED
11:06:39.092 -> [  2704][V][WiFiGeneric.cpp:294] _arduino_event_cb(): STA Got New IP:192.168.68.120
11:06:39.092 -> [  2705][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 7 - STA_GOT_IP
11:06:39.092 -> [  2708][D][WiFiGeneric.cpp:857] _eventCallback(): STA IP: 192.168.68.120, MASK: 255.255.255.0, GW: 192.168.68.1
11:06:39.557 -> Connected to XXXXXXXX
11:06:39.557 -> 
11:06:39.557 -> Starting connection to server...
11:06:39.557 -> [  3163][V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 224275
11:06:39.557 -> [  3165][V][ssl_client.cpp:65] start_ssl_client(): Starting socket
11:06:39.604 -> [  3224][V][ssl_client.cpp:141] start_ssl_client(): Seeding the random number generator
11:06:39.604 -> [  3225][V][ssl_client.cpp:150] start_ssl_client(): Setting up the SSL/TLS structure...
11:06:39.651 -> [  3228][V][ssl_client.cpp:166] start_ssl_client(): Loading CA cert
11:06:39.701 -> [  3302][V][ssl_client.cpp:234] start_ssl_client(): Setting hostname for TLS session...
11:06:39.701 -> [  3302][V][ssl_client.cpp:249] start_ssl_client(): Performing the SSL/TLS handshake...
11:06:40.591 -> [  4181][V][ssl_client.cpp:270] start_ssl_client(): Verifying peer X.509 certificate...
11:06:40.591 -> [  4181][V][ssl_client.cpp:279] start_ssl_client(): Certificate verified.
11:06:40.591 -> [  4184][V][ssl_client.cpp:294] start_ssl_client(): Free internal heap after TLS 178451
11:06:40.591 -> Connected to server!
11:06:40.591 -> [  4202][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 48 bytes...
11:06:40.591 -> [  4203][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:06:40.591 -> [  4210][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 19 bytes...
11:06:40.637 -> [  4217][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:06:40.637 -> [  4225][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 17 bytes...
11:06:40.637 -> [  4233][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:06:40.637 -> [  4240][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:06:40.732 -> HTTP/1.1 200 OK
11:06:40.732 -> Server: nginx
11:06:40.732 -> Date: Thu, 16 Sep 2021 09:06:40 GMT
11:06:40.732 -> Content-Type: application/octet-stream
11:06:40.777 -> Content-Length: 10485760
11:06:40.777 -> Last-Modified: Tue, 29 Jun 2021 14:45:37 GMT
11:06:40.777 -> Connection: close
11:06:40.777 -> ETag: "60db3211-a00000"
11:06:40.777 -> Accept-Ranges: bytes
11:06:40.777 -> 
11:06:40.777 -> headers received
11:06:40.777 -> 0
11:06:40.777 -> 500
11:06:40.777 -> 1000
11:06:40.777 -> 1500
11:06:40.823 -> 2000
11:06:40.823 -> 2500
11:06:40.823 -> 3000
11:06:40.823 -> 3500
11:06:40.870 -> 4000
11:06:40.870 -> 4500
11:06:40.870 -> 5000
11:06:40.870 -> 5500
11:06:40.870 -> 6000
11:06:40.917 -> 6500
11:06:40.917 -> 7000
11:06:40.917 -> 7500
11:06:40.917 -> 8000
11:06:40.917 -> 8500
11:06:40.964 -> 9000
11:06:40.964 -> 9500
11:06:40.964 -> 10000
11:06:40.964 -> 10500
11:06:41.011 -> 11000
11:06:41.011 -> 11500
11:06:41.011 -> 12000
11:06:41.011 -> 12500
11:06:41.011 -> 13000
11:06:41.057 -> 13500
11:06:41.057 -> 14000
11:06:41.057 -> 14500
11:06:41.057 -> 15000
11:06:41.057 -> 15500
11:06:41.105 -> 16000
11:06:41.105 -> 16135
11:06:41.105 -> [  4718][V][ssl_client.cpp:302] stop_ssl_socket(): Cleaning SSL connection.
11:06:41.105 -> Finished: read bytes: 16134


Result for 10MB file : https://proof.ovh.net/files/10Mb.dat

Code: Select all

11:11:20.713 -> Attempting to connect to SSID: XXXXXXXX
11:11:20.713 -> [    52][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 0 - WIFI_READY
11:11:20.806 -> [   139][V][WiFiGeneric.cpp:272] _arduino_event_cb(): STA Started
11:11:20.806 -> [   141][V][WiFiGeneric.cpp:96] set_esp_interface_ip(): Configuring Station static IP: 0.0.0.0, MASK: 0.0.0.0, GW: 0.0.0.0
11:11:20.806 -> [   141][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 2 - STA_START
11:11:20.853 -> ...[  2283][V][WiFiGeneric.cpp:284] _arduino_event_cb(): STA Connected: SSID: SFR-2598, BSSID: 68:ff:7b:c7:82:62, Channel: 3, Auth: WPA2_PSK
11:11:22.965 -> [  2284][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 4 - STA_CONNECTED
11:11:23.386 -> [  2740][V][WiFiGeneric.cpp:294] _arduino_event_cb(): STA Got New IP:192.168.68.120
11:11:23.432 -> [  2741][D][WiFiGeneric.cpp:808] _eventCallback(): Arduino Event: 7 - STA_GOT_IP
11:11:23.432 -> [  2744][D][WiFiGeneric.cpp:857] _eventCallback(): STA IP: 192.168.68.120, MASK: 255.255.255.0, GW: 192.168.68.1
11:11:23.852 -> Connected to XXXXXXXX
11:11:23.852 -> 
11:11:23.852 -> Starting connection to server...
11:11:23.852 -> [  3164][V][ssl_client.cpp:59] start_ssl_client(): Free internal heap before TLS 224275
11:11:23.852 -> [  3166][V][ssl_client.cpp:65] start_ssl_client(): Starting socket
11:11:23.899 -> [  3224][V][ssl_client.cpp:141] start_ssl_client(): Seeding the random number generator
11:11:23.899 -> [  3225][V][ssl_client.cpp:150] start_ssl_client(): Setting up the SSL/TLS structure...
11:11:23.899 -> [  3229][V][ssl_client.cpp:166] start_ssl_client(): Loading CA cert
11:11:23.994 -> [  3302][V][ssl_client.cpp:234] start_ssl_client(): Setting hostname for TLS session...
11:11:23.994 -> [  3303][V][ssl_client.cpp:249] start_ssl_client(): Performing the SSL/TLS handshake...
11:11:24.828 -> [  4178][V][ssl_client.cpp:270] start_ssl_client(): Verifying peer X.509 certificate...
11:11:24.874 -> [  4178][V][ssl_client.cpp:279] start_ssl_client(): Certificate verified.
11:11:24.874 -> [  4181][V][ssl_client.cpp:294] start_ssl_client(): Free internal heap after TLS 178431
11:11:24.874 -> Connected to server!
11:11:24.874 -> [  4199][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 49 bytes...
11:11:24.874 -> [  4200][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:11:24.874 -> [  4207][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 19 bytes...
11:11:24.874 -> [  4214][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:11:24.923 -> [  4222][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 17 bytes...
11:11:24.923 -> [  4230][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:11:24.923 -> [  4237][V][ssl_client.cpp:332] send_ssl_data(): Writing HTTP request with 2 bytes...
11:11:25.062 -> HTTP/1.1 200 OK
11:11:25.062 -> Server: nginx
11:11:25.062 -> Date: Thu, 16 Sep 2021 09:11:24 GMT
11:11:25.062 -> Content-Type: application/octet-stream
11:11:25.062 -> Content-Length: 104857600
11:11:25.062 -> Last-Modified: Tue, 29 Jun 2021 14:45:37 GMT
11:11:25.062 -> Connection: close
11:11:25.062 -> ETag: "60db3211-6400000"
11:11:25.062 -> Accept-Ranges: bytes
11:11:25.062 -> 
11:11:25.062 -> headers received
11:11:25.062 -> 0
11:11:25.062 -> 500
11:11:25.062 -> 1000
11:11:25.109 -> 1500
11:11:25.109 -> 2000
11:11:25.109 -> 2500
11:11:25.109 -> 3000
11:11:25.109 -> 3500
11:11:25.156 -> 4000
11:11:25.156 -> 4500
11:11:25.156 -> 5000
11:11:25.156 -> 5500
11:11:25.202 -> 6000
11:11:25.202 -> 6500
11:11:25.202 -> 7000
11:11:25.202 -> 7500
11:11:25.202 -> 8000
11:11:25.251 -> 8500
11:11:25.251 -> 9000
11:11:25.251 -> 9500
11:11:25.251 -> 10000
11:11:25.251 -> 10500
11:11:25.297 -> 11000
11:11:25.297 -> 11500
11:11:25.297 -> 12000
11:11:25.297 -> 12500
11:11:25.343 -> 13000
11:11:25.343 -> 13500
11:11:25.343 -> 14000
11:11:25.343 -> 14500
11:11:25.343 -> 15000
11:11:25.389 -> 15500
11:11:25.389 -> 16000
11:11:25.389 -> 16133
11:11:25.389 -> [  4736][V][ssl_client.cpp:302] stop_ssl_socket(): Cleaning SSL connection.
11:11:25.389 -> Finished: read bytes: 16132


Who is online

Users browsing this forum: Google [Bot] and 77 guests