Invalid characters on data received from MQTT

[JulianMauro]

Hi! I have configured an MQTT client (I'm using MQTT V5) on an ESP32-S3 that sends and receives data to/from multiple topics. The chip connects to the Internet via WiFi or LPWA (CAT-M 4G).

However, when receiving data from the different topics, sent from a Java client, the data is received with strange characters at the end of each string. I don't understand why this happens. At first I thought it might be a memory related error, but the invalid characters are always the same, no matter the content or the topic.

Also, as can be seen in the code provided, this invalid-characters-problem also affects the topic. I managed to solve it by removing the last 2 characters before doing the strcmp, however this shouldn't be necessary. Thanks in advance for any help!

Things I try to verify where the error could be:

- Tried with 2 different brokers (we use Cloudflare Pub/Sub but tried with a self-hosted instance. Same error)
- Send a simple message (Same error)
- Use a Python Script instead of the Java Script (Same error)

This is what the ESP32 receives:

Code: Select all

[127925 ms] Data: {"time":"2025-01-04T22:52:02.685-03","update":{"version":"1.5","model":"some-model","url":"https://redacted-for-privacy.com/update"}}ÁÄ
‰›¢ˇç®ùÓÖfiÇ¡7y…⁄î9fi⁄lSU™≠€âw

This is the code I am using to both connect to the MQTT Broker and the callback to receive the messages:

Code: Untitled.c Select all

#include <stdio.h>

#include <stdint.h>

#include <string.h>

#include "esp_system.h"

#include "esp_event.h"

#include "esp_netif.h"

#include "mqtt_client.h"

#include "esp_crt_bundle.h"



#include "mqtt.h"

#include "log.h"



#define MQTT_CONNECTED_BIT  (BIT0)

#define MQTT_FAIL_BIT       (BIT1)

#define MAX_RETRY           (20)



#define MQTT_UPDATES_TOPIC           ("update/esp32")

#define MQTT_CONFIGURATION_TOPIC     ("configuration/esp32")



static EventGroupHandle_t event_group;

static esp_mqtt_client_handle_t client;

static int retry_count = 0;



esp_err_t mqtt_connect(mqtt_config_t config);



static void event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data);



esp_err_t mqtt_connect(mqtt_config_t config)

{

    esp_err_t err;



    event_group = xEventGroupCreate();



    esp_mqtt_client_config_t mqtt_configuration = {

        .broker = {

            .address.uri = config.uri,

            .verification.crt_bundle_attach = esp_crt_bundle_attach // Necessary for TLS

        },

        .credentials ={               

            .username= "test",

            .client_id= config.clientid,

            .authentication.password = config.jwt

        },

        .task = {

            .stack_size = 12000,

            .priority = 16,

        },

        .session = {

            .protocol_ver = MQTT_PROTOCOL_V_5

        }

    };



    client = esp_mqtt_client_init(&mqtt_configuration);

    if (client == NULL)

    {

        SD_LOG("MQTT Client is NULL");

    }

    

    ASSERT_ON_NULL(client);

    esp_mqtt_client_register_event(client, ESP_EVENT_ANY_ID, event_handler, client);

    err = esp_mqtt_client_start(client);



    if (err != ESP_OK)

    {

        return err;

    }



    EventBits_t res = xEventGroupWaitBits(event_group, MQTT_CONNECTED_BIT | MQTT_FAIL_BIT, pdFALSE, pdFALSE, portMAX_DELAY);

    

    if ((res & MQTT_CONNECTED_BIT) != 0)

    {

        err = ESP_OK;



    } else if ((res & MQTT_FAIL_BIT) != 0) {

        err = ESP_ERR_TIMEOUT;

    }



    return err;

}



void event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data)

{

    esp_mqtt_event_handle_t event = event_data;



    switch ((esp_mqtt_event_id_t)event_id)

    {

    case MQTT_EVENT_CONNECTED:



        esp_mqtt_client_subscribe(event->client, MQTT_UPDATES_TOPIC, 0);

        esp_mqtt_client_subscribe(event->client, MQTT_CONFIGURATION_TOPIC, 0);



        retry_count = 0;

        

        xEventGroupSetBits(event_group, MQTT_CONNECTED_BIT);



        break;

    

    case MQTT_EVENT_DISCONNECTED:



        xEventGroupSetBits(event_group, MQTT_FAIL_BIT);



        break;

        

    case MQTT_EVENT_DATA:



        int topic_len = strlen(event->topic);

        event->topic[topic_len - 2] = '\0'; // This is necessary because the topic contains 2 gibrish characters at the end. Idk why.



        if (strcmp(event->topic, MQTT_UPDATES_TOPIC) == 0)

        {

            SD_LOG("Topic is updates");

            SD_LOG("Data: %s", event->data);



        } else if (strcmp(event->topic, MQTT_CONFIGURATION_TOPIC) == 0) {

        

            SD_LOG("Topic is configuration");

            SD_LOG("Data: %s", event->data);

        }



        break;



    case MQTT_EVENT_ERROR:



        if (event->error_handle->error_type == MQTT_ERROR_TYPE_CONNECTION_REFUSED)

        {

            SD_LOG("Cannot connect to MQTT. Error: %s", mqtt_connection_return_code_error[event->error_handle->connect_return_code]);

            xEventGroupSetBits(event_group, MQTT_FAIL_BIT);

        }



        break;



    default:

        break;

    }

}

[chegewara]

There is reason it works this way, and you should make use of esp_mqtt_event_handle_t structure values
https://docs.espressif.com/projects/esp ... tt_event_t

You can find in there:
- char *topic
- int topic_len
- char *data
- int data_len

[JulianMauro]

There is reason it works this way, and you should make use of esp_mqtt_event_handle_t structure values
https://docs.espressif.com/projects/esp ... tt_event_t

You can find in there:
- char *topic
- int topic_len
- char *data
- int data_len

Hi, thanks for your reply. In the code I attached, it is possible to see that I do use the esp_mqtt_event_handle_t structure. I cast it from event_data and then I access it when needed (event->topic for the topic and event->data for the data received). Still, the error persists.

Regards.

[chegewara]

Yes, i saw your code, but you do this in your code which is wrong, because topic and data are not null terminated:

Code: Select all

int topic_len = strlen(event->topic);

Thats why you have in that structure topic_len and data_len values.

[JulianMauro]

Yes, i saw your code, but you do this in your code which is wrong, because topic and data are not null terminated:

Code: Select all

int topic_len = strlen(event->topic);

Thats why you have in that structure topic_len and data_len values.

Hi, thanks for the quick response, and thanks for pointing that out. I have modified the code as follows:

Code: Select all

event->topic[event->topic_len + 1] = '\0';
event->data[event->data_len + 1] = '\0';

I cannot test it right now but I think that now it should work.

Regards

[nopnop2002]

Code: Select all

//int topic_len = strlen(event->topic);
//SD_LOG("Data: %s", event->data);

int topic_len = event->topic_len;
SD_LOG("DATA: %.*s", event->data_len, event->data);