SMTP example fails with MBEDTLS_ERR_SSL_INVALID_RECORD -0x7200

[leenowell]

Hi,

I have taken the SMTP example and added the code to my project so that when a trigger is activated it will send me an email. In order to do this, I have wrapped the example code with a C++ class and have separated the smtp_client_task function into 2 methods - one to do the initial connection and then a second one to send the email when needed. I then have a function which connects to the SMTP server then goes into an endless loop looking for events on a queue and when it receives one send the email.

If I send the email immediately after the initial connection (i.e. before the loop) then it seems to work fine but when I send the email as the result of the trigger I get a MBEDTLS_ERR_SSL_INVALID_RECORD (-0x7200) error when I try to write the first field (From). I am wondering whether I haven't split the smtp_client_task correctly? The following are the 2 methods in question

Code: Select all

void EmailClient::SMTPConnect()
{

    char *buf = NULL;
    unsigned char base64_buffer[128];
    int ret, len;
    size_t base64_len;

    mbedtls_entropy_context entropy;
    mbedtls_ctr_drbg_context ctr_drbg;
    mbedtls_x509_crt cacert;
    mbedtls_ssl_config conf;

    mbedtls_ssl_init(&m_SSL);
    mbedtls_x509_crt_init(&cacert);
    mbedtls_ctr_drbg_init(&ctr_drbg);
    ESP_LOGI(TAG, "Seeding the random number generator");

    mbedtls_ssl_config_init(&conf);

    mbedtls_entropy_init(&entropy);
    if ((ret = mbedtls_ctr_drbg_seed(&ctr_drbg, mbedtls_entropy_func, &entropy,
                                     NULL, 0)) != 0) {
        ESP_LOGE(TAG, "mbedtls_ctr_drbg_seed returned -0x%x", -ret);
        goto exit;
    }

    ESP_LOGI(TAG, "Loading the CA root certificate...");

    ret = mbedtls_x509_crt_parse(&cacert, server_root_cert_pem_start,
                                 server_root_cert_pem_end - server_root_cert_pem_start);

    if (ret < 0) {
        ESP_LOGE(TAG, "mbedtls_x509_crt_parse returned -0x%x", -ret);
        goto exit;
    }

    ESP_LOGI(TAG, "Setting hostname for TLS session...");

    /* Hostname set here should match CN in server certificate */
    if ((ret = mbedtls_ssl_set_hostname(&m_SSL, m_sMailServer)) != 0) {
        ESP_LOGE(TAG, "mbedtls_ssl_set_hostname returned -0x%x", -ret);
        goto exit;
    }

    ESP_LOGI(TAG, "Setting up the SSL/TLS structure...");

    if ((ret = mbedtls_ssl_config_defaults(&conf,
                                           MBEDTLS_SSL_IS_CLIENT,
                                           MBEDTLS_SSL_TRANSPORT_STREAM,
                                           MBEDTLS_SSL_PRESET_DEFAULT)) != 0) {
        ESP_LOGE(TAG, "mbedtls_ssl_config_defaults returned -0x%x", -ret);
        goto exit;
    }

    mbedtls_ssl_conf_authmode(&conf, MBEDTLS_SSL_VERIFY_REQUIRED);
    mbedtls_ssl_conf_ca_chain(&conf, &cacert, NULL);
    mbedtls_ssl_conf_rng(&conf, mbedtls_ctr_drbg_random, &ctr_drbg);
#ifdef CONFIG_MBEDTLS_DEBUG
    mbedtls_esp_enable_debug_log(&conf, 4);
#endif

    if ((ret = mbedtls_ssl_setup(&m_SSL, &conf)) != 0) {
        ESP_LOGE(TAG, "mbedtls_ssl_setup returned -0x%x", -ret);
        goto exit;
    }

    mbedtls_net_init(&m_ServerFD);

    ESP_LOGI(TAG, "Connecting to %s:%s...", m_sMailServer, m_sPortNumber);

    if ((ret = mbedtls_net_connect(&m_ServerFD, m_sMailServer,
                                   m_sPortNumber, MBEDTLS_NET_PROTO_TCP)) != 0) {
        ESP_LOGE(TAG, "mbedtls_net_connect returned -0x%x", -ret);
        goto exit;
    }

    ESP_LOGI(TAG, "Connected.");

    mbedtls_ssl_set_bio(&m_SSL, &m_ServerFD, mbedtls_net_send, mbedtls_net_recv, NULL);

    buf = (char *) calloc(1, BUF_SIZE);
    if (buf == NULL) {
        ESP_LOGE(TAG, "calloc failed for size %d", BUF_SIZE);
        goto exit;
    }
#if SERVER_USES_STARTSSL
    /* Get response */
    ret = writeGetResponse((unsigned char *) buf, 0);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

    ESP_LOGI(TAG, "Writing EHLO to server...");
    len = snprintf((char *) buf, BUF_SIZE, "EHLO %s\r\n", "ESP32");
    ret = writeGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

    ESP_LOGI(TAG, "Writing STARTTLS to server...");
    len = snprintf((char *) buf, BUF_SIZE, "STARTTLS\r\n");
    ret = writeGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

    ret = performTLSHandshake();
    if (ret != 0) {
        goto exit;
    }

#else /* SERVER_USES_STARTSSL */
    ret = performTLSHandshake(&m_SSL);
    if (ret != 0) {
        goto exit;
    }

    /* Get response */
    ret = writeSSLGetResponse(&m_SSL, (unsigned char *) buf, 0);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);
    ESP_LOGI(TAG, "Writing EHLO to server...");

    len = snprintf((char *) buf, BUF_SIZE, "EHLO %s\r\n", "ESP32");
    ret = writeSSLGetResponse(&m_SSL, (unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

#endif /* SERVER_USES_STARTSSL */

    /* Authentication */
    ESP_LOGI(TAG, "Authentication...");

    ESP_LOGI(TAG, "Write AUTH LOGIN");
    len = snprintf( (char *) buf, BUF_SIZE, "AUTH LOGIN\r\n" );
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 399, exit);

    ESP_LOGI(TAG, "Write USER NAME");
    ret = mbedtls_base64_encode((unsigned char *) base64_buffer, sizeof(base64_buffer),
                                &base64_len, (unsigned char *) m_sUserName, strlen(m_sUserName));
    if (ret != 0) {
        ESP_LOGE(TAG, "Error in mbedtls encode! ret = -0x%x", -ret);
        goto exit;
    }
    len = snprintf((char *) buf, BUF_SIZE, "%s\r\n", base64_buffer);
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 300, 399, exit);

    ESP_LOGI(TAG, "Write PASSWORD");
    ret = mbedtls_base64_encode((unsigned char *) base64_buffer, sizeof(base64_buffer),
                                &base64_len, (unsigned char *) m_sPassword, strlen(m_sPassword));
    if (ret != 0) {
        ESP_LOGE(TAG, "Error in mbedtls encode! ret = -0x%x", -ret);
        goto exit;
    }
    len = snprintf((char *) buf, BUF_SIZE, "%s\r\n", base64_buffer);
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 399, exit);

    ret = 0; // all OK if we get this far

exit:
	if (ret != 0) {
		mbedtls_strerror(ret, buf, 100);
		ESP_LOGE(TAG, "Last error in connect was: -0x%x - %s", -ret, buf);
	}

	putchar('\n'); /* Just a new line */
	if (buf) {
		free(buf);
	}
}

void EmailClient::SendEmail(const char *sSenderEmailAddr, const char *sSenderName, const char *sRecipientEmailAddr, const char *sSubject, const char *sMessage)
{

    char *buf = (char *) calloc(1, BUF_SIZE);
    unsigned char base64_buffer[128];
    int ret, len;
    size_t base64_len;
    const uint8_t *offset;

	/* Compose email */
    ESP_LOGI(TAG, "Write MAIL FROM");
    len = snprintf((char *) buf, BUF_SIZE, "MAIL FROM:<%s>\r\n", sSenderEmailAddr);
 // LEE DEBUG
 //   int nBuffLen = strlen((const char*)buf);
//    ESP_LOGW("LEE DEBUG", "SendEmail len[%d], strlen[%d]", len, nBuffLen);
/// END LEE DEBUG
// you could also try making buff and unsigned char *
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

    ESP_LOGI(TAG, "Write RCPT");
    len = snprintf((char *) buf, BUF_SIZE, "RCPT TO:<%s>\r\n", sRecipientEmailAddr);
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);

    ESP_LOGI(TAG, "Write DATA");
    len = snprintf((char *) buf, BUF_SIZE, "DATA\r\n");
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 300, 399, exit);

    ESP_LOGI(TAG, "Write Content");
    /* We do not take action if message sending is partly failed. */
/*    len = snprintf((char *) buf, BUF_SIZE,
                   "From: %s\r\nSubject: mbed TLS Test mail\r\n"
                   "To: %s\r\n"
                   "MIME-Version: 1.0 (mime-construct 1.9)\n",
                   "ESP32 SMTP Client", sRecipient);
*/
    len = snprintf((char *) buf, BUF_SIZE,
                   "From: %s\r\nSubject: %s\r\n"
                   "To: %s\r\n"
                   "MIME-Version: 1.0 (mime-construct 1.9)\n",
				   sSenderName, sSubject, sRecipientEmailAddr );

    /**
     * Note: We are not validating return for some ssl_writes.
     * If by chance, it's failed; at worst email will be incomplete!
     */
    ret = writeSSLData((unsigned char *) buf, len);

    /* Multipart boundary */
    len = snprintf((char *) buf, BUF_SIZE,
                   "Content-Type: multipart/mixed;boundary=XYZabcd1234\n"
                   "--XYZabcd1234\n");
    ret = writeSSLData((unsigned char *) buf, len);

    /* Text */
/*    len = snprintf((char *) buf, BUF_SIZE,
                   "Content-Type: text/plain\r\n\r\n"
                   "This is a simple test mail from the SMTP client example.\r\n"
                   "\r\n"
                   "Enjoy!\n\n--XYZabcd1234\n");
*/
    len = snprintf((char *) buf, BUF_SIZE,
                   "Content-Type: text/plain\r\n\r\n"
                   "%s\n\n--XYZabcd1234\n", sMessage);
    ret = writeSSLData((unsigned char *) buf, len);

    /* Attachment */
    len = snprintf((char *) buf, BUF_SIZE,
                   "Content-Type: image/png; name=\"esp_logo.png\"\r\n"
                   "Content-Disposition: attachment; filename=\"esp_logo.png\"\r\n"
            	   "Content-Transfer-Encoding: base64\r\n\r\n");

    ret = writeSSLData((unsigned char *) buf, len);

    /* Image contents... */
    offset = esp_logo_png_start;
    while (offset < esp_logo_png_end - 1) {
        int read_bytes = MIN(((sizeof (base64_buffer) - 1) / 4) * 3, esp_logo_png_end - offset - 1);
        ret = mbedtls_base64_encode((unsigned char *) base64_buffer, sizeof(base64_buffer),
                                    &base64_len, (unsigned char *) offset, read_bytes);
        if (ret != 0) {
            ESP_LOGE(TAG, "Error in mbedtls encode! ret = -0x%x", -ret);
            goto exit;
        }
        offset += read_bytes;
        len = snprintf((char *) buf, BUF_SIZE, "%s\r\n", base64_buffer);
        ret = writeSSLData((unsigned char *) buf, len);
    }

    len = snprintf((char *) buf, BUF_SIZE, "\n--XYZabcd1234\n");
    ret = writeSSLData((unsigned char *) buf, len);

    len = snprintf((char *) buf, BUF_SIZE, "\r\n.\r\n");
    ret = writeSSLGetResponse((unsigned char *) buf, len);
    VALIDATE_MBEDTLS_RETURN(ret, 200, 299, exit);
    ESP_LOGI(TAG, "Email sent!");

    ret = 0; /* No errors */

exit:
    if (ret != 0) {
        mbedtls_strerror(ret, buf, 100);
        ESP_LOGE(TAG, "Last error was: -0x%x - %s", -ret, buf);
    }

    putchar('\n'); /* Just a new line */
    if (buf) {
        free(buf);
    }
}

I have also checked the following variables in the sdkconfig file as I saw some people having the issue because the buffers were too small but I believe these are set to max?

Code: Select all

CONFIG_MBEDTLS_ASYMMETRIC_CONTENT_LEN=y
CONFIG_MBEDTLS_SSL_IN_CONTENT_LEN=16384
CONFIG_MBEDTLS_SSL_OUT_CONTENT_LEN=4096

Thanks in advance for your help with this

Lee.

[MicroController]

Multiple possibilities as to what might be going wrong. One of them:

one [method] to do the initial connection and then a second one to send the email when needed

Note that SMTP (unlike IMAP) is not intended for long-lived connections. It may well be that the SMTP server closes the connection after just some 10-30 seconds of inactivity.

(And the mbedTLS layer tends to return 'confusing' error codes whenever it does not get the data it expects from the transport layers.)

[leenowell]

Thanks very much for getting back to me. I have tried to do the connect and send each time and that seems to work so looks like you were correct. Thanks very much for your help.