Intermittent Crash when implementing Example 19 - Dynamic Task with AutoDisable, and Serial stops working

[Jimbo181]

Hi. I have inserted a watchdog system into my ESP32 project based upon the example 19 code. I am using PlatfomIO, and V3.7.0 of your library.
Couple of issues have arisen:

1. If I include the #define line #define _TASK_SLEEP_ON_IDLE_RUN then my RS232 reader no longer functions. Any workarounds? This is currenly not enabled - possible cause for 2 below?
2. I am getting an intermittent exception and reboot. Happens at least every couple of minutes, normally after maybe 1 minute.
   The Core dump indicates the problem occurs on the scheduler.execute line in the Loop function, Could you possibly look at the code and see if I did something wrong?
   Note 1: If I don't allow any connections, so no new tasks are created, the crash seems to stop.
   Note 2: Perhaps the initialisation of the structure with the task pointer is problematic?
   Many thanks, Jim
   Code extracts below...

`#include <Arduino.h>
#include <FastLED.h>
#include <WiFiUDP.h>
#include <ETH.h>
#include <WiFi.h>
#include <ESPmDNS.h>
#include <AsyncUDP.h>
#include
#include
#include <ArduinoJson.h>

//Task Scheduler flags for watchdog stuff
#define _TASK_WDT_IDS // To enable task unique IDs
//#define _TASK_SLEEP_ON_IDLE_RUN // Compile with support for entering IDLE SLEEP state for 1 ms if not tasks are scheduled to run
#define _TASK_LTS_POINTER // Compile with support for Local Task Storage pointer
#define _TASK_SELF_DESTRUCT // Enable tasks to "self-destruct" after disable
#include <TaskScheduler.h>
#include <string.h>
#include
#include <SPI.h>
#include <Wire.h>
#include <LiquidCrystal_I2C.h>
#include <HardwareSerial.h>
#include <include.h>
#include <funcDecs.h>

HardwareSerial SerialPort(2); // use UART2`

`//Global Variables
Scheduler userScheduler; // to control your personal task
Task taskTempoAction( TASK_MILLISECOND * 60000/tempoBPM, TASK_FOREVER, &tempoAction); // Scheduler for running Tempo Actions
Task repeatLastMessage(TASK_MILLISECOND * MSG_REPEAT_INTERVAL, TASK_ONCE, &repeatMessage);

struct macEntry {
String macAddress;
String drumName;
bool connected;
Task *udpWatchdog;
};

using std::vector;
vector macTable = { /******* MAC ADDRESS MUST BE ALL CAPS!!! ********/
{"78:21:84:9A:C3:90","Test ESP 1",false},
{"A0:B7:65:4D:44:9C","Test ESP 2",false}
};`

`void setup()
{
Serial.begin(115200); // Start the debug serial interface UART

SerialPort.begin (SERIAL_BAUD, SERIAL_8N1,PIN_RX,PIN_TX); // Rx on 13, Tx on 15 for inter ESP UART Comms

WiFi.onEvent(WiFiEvent);

WiFi.begin();
ETH.begin();
Serial.printf("ETH: Configuring network: %s, %s\n",rxIp.toString().c_str(), rxSubnet.toString().c_str());

// Configures static IP address
if (!ETH.config(rxIp, rxGateway, rxSubnet, primaryDNS, secondaryDNS)) {
Serial.println("ETH: STA Failed to configure");
}
Serial.printf("ETH: Starting network: %s, %s\n",rxIp.toString().c_str(), rxSubnet.toString().c_str());

if (MDNS.begin("esp32")) {
Serial.println("MDNS responder started");
}

userScheduler.addTask(taskTempoAction); // sends linked action messages from song playack
userScheduler.addTask(repeatLastMessage); // message repeat timer

taskTempoAction.enable();

Serial.printf("There are %d buttons\n",numButtons);
Serial.printf("There are %d static states\n",numStatics);
Serial.printf("There are %d hsv states\n",numHSVs);
Serial.printf("Tempo is %d BPM\n",tempoBPM);

#ifdef DEBUG_PRINT
Serial.println("Debug Printing is ON");
#else
Serial.println("Debug Printing is OFF");
#endif

if(rxUdp.listen(RX_PORT))
{
Serial.printf("Listening for UDP on Port %04d, IP:",RX_PORT);
Serial.print(WiFi.localIP());
Serial.print(" MAC: ");
Serial.println(WiFi.macAddress());

rxUdp.onPacket([](AsyncUDPPacket packet)
{
  
  Serial.print("UDP Packet Type: ");
  Serial.print(packet.isBroadcast()?"Broadcast":packet.isMulticast()?"Multicast":"Unicast");
  Serial.print(", From: ");
  Serial.print(packet.remoteIP().toString().c_str());
  Serial.print(":");
  Serial.print(packet.localPort());
  Serial.print(", To: ");
  Serial.print(packet.localIP());
  Serial.print(":");
  Serial.print(packet.localPort());
  Serial.print(", Length: ");
  Serial.print(packet.length());
  //Serial.print(", Data: ");
  //Serial.write(packet.data(), packet.length());
  Serial.println();
  
  char* tempStr = (char*) malloc (packet.length()+1);
  memcpy (tempStr, packet.data(), packet.length());
  tempStr[packet.length()] = '\0';

  String message = String(tempStr);
  free(tempStr);
  Serial.print("Received UDP Msg: ");
  Serial.println(message);
  Serial.println();
  String MACAddress = message.substring(11);

  bool found = false;
  for(int i = 0; i< macTable.size(); i++)
  {
    if(macTable[i].macAddress == MACAddress)
    {
      //Serial.printf("%s has said Hi\n",macTable[i].drumName.c_str());
      if(macTable[i].connected == false) // new connection, so start the timeout timer
      {
        macTable[i].udpWatchdog = new Task(20000, TASK_ONCE, watchdog, &userScheduler, false, OnEnable, OnDisable, true);
        Serial.print("MEM: ");
        Serial.println(ESP.getFreeHeap());
        macTable[i].udpWatchdog->setId(i);
        userScheduler.addTask(*(macTable[i].udpWatchdog)); // Add the watchdog to the scheduler
        macTable[i].udpWatchdog->restartDelayed(20000);
        //Serial.printf("Task ID %d has been added to the scheduler and started for MAC entry %d\n",macTable[i].udpWatchdog->getId(),i);
        macTable[i].connected = true;
        Serial.printf("%s status: Connected\n\n",macTable[i].drumName.c_str());
        numConnected++;
        
        String lineTemp = "Hello ";
        String line1 = lineTemp + macTable[i].drumName;
        String line2 = "Total: ";
        line2 = line2 + numConnected;
        updateLCD(line1,line2);
        
      } else {                          // else already connected so just reset the timeout
        macTable[i].udpWatchdog->restartDelayed(20000);
        //Serial.printf("Restarted watchdog for ID %d\n",i);
      }
      found = true;
      break;
    }
  }
  if (!found)
    Serial.printf("unknown ESP32, MAC %s has said Hi\n",MACAddress.c_str());
  

});

}

updateLCD("Started");
}

void loop() {

userScheduler.execute(); // run the task scheduler

#ifdef MANUAL_BUTTONS
debounceSwitch();
#endif

#ifdef MANUAL_BUTTONS
EVERY_N_MILLISECONDS(500)
{
if (mode == off)
{
pauseLED = !pauseLED;
updateButtonLEDs();
}
}
#endif

//EVERY_N_MILLISECONDS(200)
// lcd.scrollDisplayLeft();

if (SerialPort.available()) // theres a message coming in from the Web Server ESP
{
StaticJsonDocument<800> doc; // BEWARE!! Making this too big seems to cause crashing...
`

load of other stuff for the Serial read then...

`void watchdog(void)
{
Task &thisTask = userScheduler.currentTask();
int thisId = thisTask.getId();
//Serial.printf("Watchdog timer callback for ID %d has hapened\n",thisId);
macTable[thisId].connected = false;
macTable[thisId].udpWatchdog->disable();
userScheduler.deleteTask(thisTask);
//Serial.printf("Watchdog timer for ID %d has been deleted\n\n",thisId);
Serial.printf("%s has disconnected\n\n",macTable[thisId].drumName.c_str());
numConnected--;

String lineTemp = "Lost: ";
String line1 = lineTemp + macTable[thisId].drumName;
String line2 = "Total: ";
line2 = line2 + numConnected;
updateLCD(line1,line2);

}

bool OnEnable()
{
// to-do: think of something to put in here.
return true;
}

void OnDisable()
{
// do something in here when a task is disabled (ends)
}`

Stack Dump as follows:

`Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.

Core 1 register dump:
PC : 0x400d344e PS : 0x00060130 A0 : 0x800d7b79 A1 : 0x3ffb1f80
A2 : 0x00000001 A3 : 0x3ffc2e18 A4 : 0x00000000 A5 : 0x00000000
A6 : 0x3ffc38ec A7 : 0x3ffb1f3c A8 : 0x00000000 A9 : 0x3ffb1f60
A10 : 0x000876eb A11 : 0x00004e20 A12 : 0x00004e20 A13 : 0x3ffb26e0
A14 : 0x00000003 A15 : 0x00060023 SAR : 0x0000000a EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000008 LBEG : 0x400848e5 LEND : 0x400848ed LCOUNT : 0x00000026

Backtrace:0x400d344b:0x3ffb1f800x400d7b76:0x3ffb1fa0 0x400de441:0x3ffb2820

#0 0x400d344b:0x3ffb1f80 in Scheduler::execute() at .pio/libdeps/wt32-eth01/TaskScheduler/src/TaskScheduler.h:1345
(inlined by) Scheduler::execute() at .pio/libdeps/wt32-eth01/TaskScheduler/src/TaskScheduler.h:1275
#1 0x400d7b76:0x3ffb1fa0 in loop() at src/main.cpp:439
#2 0x400de441:0x3ffb2820 in loopTask(void*) at C:/Users/Jim/.platformio/packages/framework-arduinoespressif32/cores/esp32/main.cpp:50`

where line 439 is the userSchedule.execute(); in my loop.

Cheers, and many thanks for the Scheduler - its a great library.
Jim

[arkhipenko]

HI,

_TASK_SLEEP_ON_IDLE_RUN for esp32 is experimental - I would not use it.
I was experimenting with "light sleep" mode for esp32, but it messes up WiFi and probably other processes as well.
Don't use it.

[Jimbo181]

Yes, it was this that was stopping my RS232 serial port from being updated. Thanks - good to know it can be omitted.

[arkhipenko]

Also,
if you are not creating tasks dynamically - do not use
#define _TASK_SELF_DESTRUCT // Enable tasks to "self-destruct" after disable
This option will destroy the task on disable leaving you with a NULL pointer.

[Jimbo181]

Thanks for this,
I am actually creating the tasks dynamically, and I am using the following when a task should be deleted:

macTable[thisId].udpWatchdog->disable(); userScheduler.deleteTask(thisTask);

This is only called when the task calls the Calback function.

I dont believe that this is the failure point, because in normal circumstances, the tasks are constantly restarted using:

macTable[i].udpWatchdog->restartDelayed(20000);

It appears that something is wrong with the tasks when schedule.update{} gets called, but i am not sure what it could be.

Is my declaration of the Task * in my structure correct? Perhaps the declaration needs to reserve some space for the pointer? Do I need to set the pointer to NULL or something when the vector is initialised?:

`struct macEntry {
String macAddress;
String drumName;
bool connected;
Task *udpWatchdog;
};

using std::vector;
vector macTable = { /******* MAC ADDRESS MUST BE ALL CAPS!!! ********/
{"78:21:84:9A:C3:90","Test ESP 1",false},
{"A0:B7:65:4D:44:9C","Test ESP 2",false}
};`

Thanks, Jim
Edit: I can confirm that if no dynamic tasks are currently created, even if they have been previously created and then destroyed, the crash does not happen.

[arkhipenko]

Somehow I do not see your callback code for tempoAction and repeatMessage
Regardless,
if you use _TASK_SELF_DESTRUCT
this will become a problem:
macTable[thisId].udpWatchdog->disable(); userScheduler.deleteTask(thisTask);
disable() will delete the task and NULL the pointer, which will cause userScheduler.deleteTask(thisTask); to crash with NULL point exception.
Task deletion is a safe operation - as in it would remove itself from the scheduler chain.

So either do not use _TASK_SELF_DESTRUCT or do not call userScheduler.deleteTask(thisTask); after disable.

[arkhipenko]

Plus userScheduler.deleteTask(thisTask); only removes task from the execution chain, does not delete the task as an object.

[Jimbo181]

Hi. I have tried your suggestion, and removed the .deleteTask() line. Unfortunately this has not stopped the intermittent crashes,
My code operates as follows:
Every time a new UDP MAC address is received, a Task is dynamically created, with a new unique ID, The task is a once only, with a timeout of 20s.
From then on, as long as the device will continue to send UDP every 10s, each time it is received, the Task will be restarted.
In this way, if the device is still connected, the task will never call the callback function watchdog().
If however no UDP is received for 20s, then the associated task will timeout, watchdog() will be called, and the device will be marked as disconnected and the task will be disabled() - and I assume automatically destroyed?

The crashing only occurs, if one or more devices are 'connnected'. With the device sending UDP, the task(s) never timeout, as they are continually restarted (restartDelayed(2000) ) as the UDP is received, and therefore watchdog() is never called.

So what I'm saying is that its not the destruction of tasks that seem to be causing the issue, as they are not being disabled/destructed/deleted/timing out when the crashing occurs.

FYI, I tried recompiling without adding the two other tasks (tempoAction and repeatLastMessage() ) added to the scheduler and still crashing. I also manually changed the IDs of these two tasks to 10000 and 10001 so there would be no conflict, and again no good.

Would a stripped down version of the code to just the scheduling be of use?
Regards and many thanks, Jim

[arkhipenko]

Would a stripped down version of the code to just the scheduling be of use?

Absolutely - a minimal code that compiles and can reproduce the issue is aways best!