What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

tuskiomi
Posts: 23
Joined: Mon Aug 28, 2017 8:21 pm

What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby tuskiomi » Sun Sep 10, 2017 11:01 pm

I have a simple sketch for the arduino DE that has 3 tasks running simultaneously. On seemingly random timings, I get the following error:
Task watchdog got triggered. The following tasks did not feed the watchdog in time:
Tasks currently running:
CPU 0: ALTIMITER
CPU 1: IDLE
With no rhyme or reason as to WHEN this starts happening, it always eventually will, and when it starts, it repeats, and does not let my code's debug output print to serial.

I'm at a loss for what is even happening.

What is a watchdog?
Why does it get triggered?
What is feeding it?
what is it being fed?
What are common causes of watchdog starvation?


tuskiomi
Posts: 23
Joined: Mon Aug 28, 2017 8:21 pm

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby tuskiomi » Mon Sep 11, 2017 2:04 am

Yes. I've read that. I can't make heads or tails of it. For example:
By default, the task watchdog watches the idle tasks.
Does this mean it's also fed by idle tasks? what's an 'Idle task'? one that's called vTaskDelay?
The usual cause of idle tasks not feeding the watchdog is a higher-priority process looping without yielding to the lower-priority processes, and can be an indicator of badly-written code that spinloops on a peripheral or a task that is stuck in an infinite loop.
How does a task Yield to another? why would a HIGHER priority task yield to a LOWER priority task? This happened when I only had 2 tasks as well. With 2 cores, this should never happen, no? what are the idle tasks actually feeding the watchdogs? The two loops are using I2c. is this indicative of a non-responsive device?
Other task can elect to be watched by the task watchdog by calling esp_task_wdt_feed(). Calling this routine for the first time will register the task to the task watchdog; calling it subsequent times will feed the watchdog. If a task does not want to be watched anymore (e.g. because it is finished and will call vTaskDelete() on itself), it needs to call esp_task_wdt_delete().
So a watched task must feed the dog, but an unwatched task doesn't? what happens when the dog would normally throw a fit?
The task watchdog is built around the hardware watchdog in timer group 0.

This is the processor clock, right?
If this watchdog for some reason cannot execute the interrupt handler that prints the task data (e.g. because IRAM is overwritten by garbage or interrupts are disabled entirely) it will hard-reset the SOC.
so a dead watchdog makes the software reset?

User avatar
kolban
Posts: 1683
Joined: Mon Nov 16, 2015 4:43 pm
Location: Texas, USA

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby kolban » Mon Sep 11, 2017 2:23 am

What APIs or libraries or logic are you using to create tasks in an Arduino environment?
Free book on ESP32 available here: https://leanpub.com/kolban-ESP32

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby ESP_Sprite » Mon Sep 11, 2017 2:40 am

The watchdog timer issue you're running into is that by default, the idle task calls the watchdog as well. It's needed for the idle task to run every now and then, because it's responsible for household tasks like cleaning up TCBs of deleted tasks, so by default it feeds a task watchdog. If this watchdog triggers, it essentially means a high-priority task is using up all the CPU time, without calling (directly or indirectly) a FreeRTOS function that yields execution from that high-prio task to lower-prio tasks.

Also, the task watchdog warning you see is just that: a warning. If your code doesn't run properly, it's because of something else.

tuskiomi
Posts: 23
Joined: Mon Aug 28, 2017 8:21 pm

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby tuskiomi » Mon Sep 11, 2017 7:43 pm

ESP_Sprite wrote:The watchdog timer issue you're running into is that by default, the idle task calls the watchdog as well. It's needed for the idle task to run every now and then, because it's responsible for household tasks like cleaning up TCBs of deleted tasks, so by default it feeds a task watchdog. If this watchdog triggers, it essentially means a high-priority task is using up all the CPU time, without calling (directly or indirectly) a FreeRTOS function that yields execution from that high-prio task to lower-prio tasks.

Also, the task watchdog warning you see is just that: a warning. If your code doesn't run properly, it's because of something else.
Yes, but remember that the other CPU is IDLE. Why won't the task spool up on that core?

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby ESP_Sprite » Tue Sep 12, 2017 1:04 am

Because of the way things work, we have two idle tasks. This means that both CPUs need to be able to run the idle task every now and then.

burkulesomesh43
Posts: 132
Joined: Tue Aug 14, 2018 6:21 am
Location: India

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby burkulesomesh43 » Thu Jan 10, 2019 6:10 pm

ESP_Sprite wrote:
Mon Sep 11, 2017 1:42 am
You may want to read up on the esp-idf watchdogs: http://esp-idf.readthedocs.io/en/latest ... /wdts.html .
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?
--
Somesh Burkule

mikemoy
Posts: 599
Joined: Fri Jan 12, 2018 9:10 pm

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby mikemoy » Thu Jan 10, 2019 7:20 pm

burkulesomesh43 wrote:
Thu Jan 10, 2019 6:10 pm
ESP_Sprite wrote:
Mon Sep 11, 2017 1:42 am
You may want to read up on the esp-idf watchdogs: http://esp-idf.readthedocs.io/en/latest ... /wdts.html .
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?
You might want to re-visit how your coding the application. you should not be starving the watchdog. In any even you can change its settings in menuconfig

User avatar
fly135
Posts: 606
Joined: Wed Jan 03, 2018 8:33 pm
Location: Orlando, FL

Re: What is a watchdog, why does it need to be feed, what is it fed, and why does it stop my code from running properly?

Postby fly135 » Thu Jan 10, 2019 9:47 pm

burkulesomesh43 wrote:
Thu Jan 10, 2019 6:10 pm
Is there any effect if we disable the watchdogs ??
I am using upto 10 tasks in my code but giving me an TG1WDT RESET ERROR. So I want to remove this error to prevent esp32 from reset. I want my esp32 live for long time.
Could you please explain if any issue by disabling watchdogs?
Is the WD resetting your ESP32? If you preform a make menuconfig, there is a setting under component config/esp32-specific called "Invoke panic handler on Task watchdog timeout". If you un-select that, then the WD timeouts will print bu not reset the chip or affect anything except for printing the statement. You can also change the time from 60 secs to a higher number.

The reason you are getting the WD statement is because the task ALTIMITER is executing with no statements to allow other lower priority tasks to execute. The IDLE task is a low priority task that tells the WD that everything is OK because the application is not hung. If you don't care about the device rebooting itself if it's hung then you don't need a WD.

The reason that ALTIMITER is not allowing the IDLE task to run in a 60 sec period is the issue here. If you know that you have 100 secs of processing in ALTIMITER before you use some mechanism to allow the task to go idle then you can just change the WD timeout to a number higher than 100. Or maybe the design of your app doesn't let tasks other than ALTIMITER to run. Only you know what your intentions are.

It's always a good idea to have a task supervise the application and make sure it is functioning. The IDLE task is the default "supervisor". When it doesn't run it lets you know with a WD. You can code up an alternative way of supervising. But if the supervisor task never gets invoked because your application is hung up, then the only way the supervisor can do the job of resetting the ESP32 is with a WD of some kind.

John A

Who is online

Users browsing this forum: No registered users and 112 guests