Problem uploading file with WiFiClientSecure

grasvezel
Posts: 10
Joined: Mon Feb 12, 2018 4:54 pm

Problem uploading file with WiFiClientSecure

Postby grasvezel » Mon Feb 12, 2018 5:38 pm

Hi,

I'm working on a datalogger. The thing is supposed to pass measurements on to a webserver. That part works. Over SSL using WiFiClientSecure. If the network or server is unavailable, I want to store data on an SD card. Works too. But, when I'm trying to upload this (text) file to the webserver over SSL, it fails. I receive only the first 5 bytes of the file (excluding the HTTP headers) and that's it. If I use WiFiClient and change the port from 443 to 80 it works fine, but I want to use SSL (core debug level set to ERROR) I get:

[E][ssl_client.cpp:28] handle_error(): UNKNOWN ERROR CODE (004E)
[E][ssl_client.cpp:30] handle_error(): MbedTLS message code: -78

If I compile with core debug level set to VERBOSE, the exact same code works fine and I am able to upload a file using SSL. Terribly slow, as there is a lot of debug info to be written. It only works with VERBOSE, if I lower the debug level to DEBUG I get the same error as pasted above.

This seems like a bug to me. I would really appreciate it if someone would be willing to look at this. Let me know if you need more input.

grasvezel
Posts: 10
Joined: Mon Feb 12, 2018 4:54 pm

Re: Problem uploading file with WiFiClientSecure

Postby grasvezel » Thu Feb 22, 2018 10:17 am

No one? Wrong category? No one interested in using the security features of the ESP32 (that would be consistent with most IoT devices out there), or just no one who has a solution?

ESP_Sprite
Posts: 1608
Joined: Thu Nov 26, 2015 4:08 am

Re: Problem uploading file with WiFiClientSecure

Postby ESP_Sprite » Thu Feb 22, 2018 12:05 pm

To be fair, you're not giving us much to go on... it would help if you could e.g. post the source code of the bit failing. If you really do think this is a SDK bug, I suggest you create an issue in the ESP32 Arduino Github, as the forum is meant as a more casual way of conversing.

grasvezel
Posts: 10
Joined: Mon Feb 12, 2018 4:54 pm

Re: Problem uploading file with WiFiClientSecure

Postby grasvezel » Thu Feb 22, 2018 2:42 pm

Thanks for pointing that out. I'm quite sure it's a bug, but I don't know if it's a bug in the SDK or in WiFiClientSecure.

The code is below. This works, but if I change WiFiClient to WiFiClientSecure (and change the port from 80 to 443) it works only if I compile the sketch with verbose debugging enabled. Without verbose debugging, I only receive the first 5 bytes of the file.

The 'file received' thingy is sent as a reply by the server to indicate the succesfull reception of the file, just to make sure not to delete it from the SD card if something went wrong.

Code: Select all

// file upload from SD
void uploadSDfile(String filename, bool delAfterUpload) {
  if(SD.begin()) {
    File file = SD.open(filename);
    if(!file) {
      addLog(LOG_LEVEL_ERROR, "SD   : Unable to open " + filename);
      file.close();
      return;
    }
    SD.end();
    String url = "/upload/?file=log&id=" + String(chipMAC);
    char* server = DEFAULT_LOG_HOST;
    WiFiClient uploadclient;
    int port = 80;
    if (!uploadclient.connect(server, port)) {
      // Don't write to log. File is opened!
      Serial.println("                    WEBCL: Connection to " + String(server) + " failed!");
    } else {
      Serial.println("                    WEBCL: Connected to " + String(server));
      uploadclient.println("POST " + url + " HTTP/1.1");
      uploadclient.println("Host: " + String(server));
      uploadclient.println("Connection: close");
      uploadclient.println("Content-type: text/plain");
      uploadclient.println("Content-length: " + String(file.size()));
      uploadclient.println();
      // send file contents to webserver
      while (file.available() && uploadclient.connected()) {
        uploadclient.print((char)file.read());
      }
      file.close();
      while (uploadclient.connected()) {
        String line = uploadclient.readStringUntil('\n');
        if(line == "*file received*") {
           Serial.println("                    UPLD : File received!");
           if(delAfterUpload) {
             Serial.println("                    UPLD : Deleting file " + filename);
             if(SD.remove(filename)) {
               addLog(LOG_LEVEL_INFO, "UPLD : File " + filename + " deleted after upload");
             }
           }
        }
      }
      uploadclient.stop();
    }
  } else {
    addLog(LOG_LEVEL_ERROR, "SD   : Unable to mount SD card");
  }
}



tele_player
Posts: 90
Joined: Sun Jul 02, 2017 3:38 am

Re: Problem uploading file with WiFiClientSecure

Postby tele_player » Sun Feb 25, 2018 2:27 am

Probably unrelated, but seems to me all access to the SD card should be between SD.begin() and SD.end().

grasvezel
Posts: 10
Joined: Mon Feb 12, 2018 4:54 pm

Re: Problem uploading file with WiFiClientSecure

Postby grasvezel » Mon Feb 26, 2018 9:15 am

It is. I had some trouble pasting the code and I messed it up.

grasvezel
Posts: 10
Joined: Mon Feb 12, 2018 4:54 pm

Re: Problem uploading file with WiFiClientSecure

Postby grasvezel » Thu Mar 08, 2018 9:29 am

I've solved this issue. Well, I didn't really solve it, but I found a workaround. After talking to copercini (the author of the lib) and doing some experimenting with adding delays at various points in the code (wich helped a bit but didn't really solve the issue), I modified my code to send larger packets. It turns out the library does not buffer bytes written until the IP packet to be sent over WiFi is full, but it sends out a packet for every .write(). So I decided to use a String as a buffer. After that modification to the code, I can now upload larger files (I tested up to about 1.5MB) over HTTPS. This is the relevant part of the code:

Code: Select all

    // upload the file in 1350 byte chunks
    while(file.available()) {
      int nextPacketSize = file.available();
      if (nextPacketSize > 1350) {
        nextPacketSize = 1350;
      }
      String buffer = "";
      for(int i=0;i<nextPacketSize;i++) {
        buffer += (char)file.read();
      }
      uploadclient.print(buffer);
    }


This now works with both WiFiClient and WiFiClientSecure.

Who is online

Users browsing this forum: No registered users and 1 guest