Insights suddenly stops getting updates

[djorr5]

Hi,

I have noticed that the Insights dashboard suddenly stops presenting values at a random point in time. I had a board connected on 29th Sept at 8:55am and was sending data through until 1st Oct at 2:46am. The board is still functioning and I can control it etc, but Insights doesn't display anymore informatino from teh 01/10/2024 onwards.

Node ID = Uo9LGDhnqiHom72seFCYdZ

I am only sending a custom parameter every 2 minutes and Wifi metrics. The wifi metrics sends every 30 seconds but I have not found a way of altering the frequency of that. Either way I should be affecting the fair use policy so not sure why it suddenly stops connection. The board is not connected to a PC so I can't see the logs and I was hoping you could check if something is happening in your backend that stops this data being displayed?

[Piyush]

Hello, sorry for not responding earlier, but we indeed see that there was no data received in Insights for a few days but it started again on 12th. Did anything change on your side making it work again?

[djorr5]

I power cycled the device.

[djorr5]

@ESP_Piyush

It suddenly stopped again. I just noticed it this morning and looking into Insights it shows it suddenly just stopped yesterday:

Screenshot 2024-10-22 092715.png (60.64 KiB) Viewed 21247 times

In esp_idf it jsut shows that wifi metrics failed to add, also the parameters aren't being sent either, however the esp_rmaker_param_update_and_report doesn't seem to throw an error.

Screenshot 2024-10-22 092957.png (65.49 KiB) Viewed 21247 times

The node is still connected to Rainmaker because I can still send control messages and they are received successfully.

Screenshot 2024-10-22 093312.png (43.26 KiB) Viewed 21247 times

Any idea what might be causing this?

[djorr5]

It happened again.

No other alerts happened around that time, just stopped all of a sudden at 12/12/2024 21:05

[Vikram]

Hi @djor, we have found a fix for the issue. It only occurs when Insights is used with the external transport, (ESP RainMaker in this context).
Do check with the below patch applied, and it should fix the issue.

Code: Untitled.c Select all

From 904eb2e57a588123c0a377ef7c57e266a0a22b40 Mon Sep 17 00:00:00 2001

From: Shardul Nalegave <shardul.nalegave@espressif.com>

Date: Tue, 24 Dec 2024 11:19:08 +0530

Subject: [PATCH 1/1] Bug fix: MQTT event handler incorrectly resets timers.



- Event handler was incorrectly resetting data send timers causing a lost message to completely block further requests.

- Fixed by moving timer reset logic to correct location.

- For further information, see: https://www.esp32.com/viewtopic.php?f=41&t=42129&sid=73d94616778e403b74fc58dcb8378a12

---

 components/esp_insights/src/esp_insights.c | 22 ++++++++++++++++------

 1 file changed, 16 insertions(+), 6 deletions(-)



diff --git a/components/esp_insights/src/esp_insights.c b/components/esp_insights/src/esp_insights.c

index 44168cd..3d8bf8e 100644

--- a/components/esp_insights/src/esp_insights.c

+++ b/components/esp_insights/src/esp_insights.c

@@ -279,25 +279,31 @@ static void insights_event_handler(void* arg, esp_event_base_t event_base,

     }

     switch(event_id) {

         case INSIGHTS_EVENT_TRANSPORT_SEND_SUCCESS:

-#if INSIGHTS_DEBUG_ENABLED

-            ESP_LOGI(TAG, "Data send success, msg_id:%d.", data ? data->msg_id : 0);

-#endif

             if (data && data->msg_id) {

                 xSemaphoreTake(s_insights_data.data_lock, portMAX_DELAY);

-                if (xTimerIsTimerActive(s_insights_data.data_send_timer) == pdTRUE) {

-                    xTimerStop(s_insights_data.data_send_timer, portMAX_DELAY);

-                }

                 if (data->msg_id == s_insights_data.data_msg_id) {

+#if INSIGHTS_DEBUG_ENABLED

+                    ESP_LOGI(TAG, "Data message send success, msg_id:%d.", data ? data->msg_id : 0);

+#endif

                     esp_diag_data_store_critical_release(s_insights_data.data_msg_len);

                     s_insights_data.data_sent = true;

                     s_insights_data.data_send_inprogress = false;

+                    if (xTimerIsTimerActive(s_insights_data.data_send_timer) == pdTRUE) {

+                        xTimerStop(s_insights_data.data_send_timer, portMAX_DELAY);

+                    }

 #if SEND_INSIGHTS_META

                 } else if (s_insights_data.meta_msg_pending && data->msg_id == s_insights_data.meta_msg_id) {

+#if INSIGHTS_DEBUG_ENABLED

+                    ESP_LOGI(TAG, "Meta message send success, msg_id:%d.", data ? data->msg_id : 0);

+#endif

                     esp_insights_meta_nvs_crc_set(s_insights_data.meta_crc);

                     s_insights_data.meta_msg_pending = false;

                     s_insights_data.data_sent = true;

 #endif /* SEND_INSIGHTS_META */

                 } else if (s_insights_data.boot_msg_id > 0 && s_insights_data.boot_msg_id == data->msg_id) {

+#if INSIGHTS_DEBUG_ENABLED

+                    ESP_LOGI(TAG, "Boot message send success, msg_id:%d.", data ? data->msg_id : 0);

+#endif

 #if CONFIG_ESP_INSIGHTS_COREDUMP_ENABLE

                     esp_core_dump_image_erase();

 #endif // CONFIG_ESP_INSIGHTS_COREDUMP_ENABLE

@@ -305,9 +311,13 @@ static void insights_event_handler(void* arg, esp_event_base_t event_base,

                 }

 #if INSIGHTS_CMD_RESP

                 else if (s_insights_data.conf_msg_id > 0 && s_insights_data.conf_msg_id == data->msg_id) {

+#if INSIGHTS_DEBUG_ENABLED

+                    ESP_LOGI(TAG, "Conf message send success, msg_id:%d.", data ? data->msg_id : 0);

+#endif

                     s_insights_data.conf_msg_id = 0;

                 }

 #endif

+

                 xSemaphoreGive(s_insights_data.data_lock);

             }

             break;

-- 

2.39.5 (Apple Git-154)

[djorr5]

@ESP_Vikram thanks for this. Sorry only getting back to it now as the issue popped up again. Let me test this out and I will get back to you.

[Piyush]

The fix is also pushed out to esp-insights GitHub project.