ESP32-CAM + Python Flask Face Recognition Server Crash (Guru Meditation Error)

[redsky]

I'm working on a project that uses face recognition with an ESP32-CAM and a Python Flask server. The server processes the image and responds with either access granted or denied. However, my ESP32 crashes with a Guru Meditation Error right after connecting to Wi-Fi and attempting to send the image.

PYTHON CODE (Flask server for face recognition):

Code: Select all

from flask import Flask, request, jsonify
import face_recognition
import cv2
import numpy as np
import os

app = Flask(__name__)

REFERENCE_IMAGES_DIR = r"C:\project\refrence images"
known_encodings = []
known_names = []

def load_reference_images():
    global known_encodings, known_names
    for file_name in os.listdir(REFERENCE_IMAGES_DIR):
        if file_name.lower().endswith(('.png', '.jpg', '.jpeg')):
            name = os.path.splitext(file_name)[0]
            file_path = os.path.join(REFERENCE_IMAGES_DIR, file_name)
            try:
                image = face_recognition.load_image_file(file_path)
                encodings = face_recognition.face_encodings(image)
                if encodings:
                    known_encodings.append(encodings[0])
                    known_names.append(name)
                    print(f"Loaded reference image: {file_name} -> {name}")
                else:
                    print(f"No face detected in reference image: {file_name}")
            except Exception as e:
                print(f"Error loading reference image {file_name}: {e}")

load_reference_images()

@app.route('/upload', methods=['POST'])
def upload():
    if 'image' not in request.files:
        return jsonify({"message": "No image data received", "access": "denied"}), 400
    
    file = request.files['image']
    if file.filename == '':
        return jsonify({"message": "No selected file", "access": "denied"}), 400

    upload_path = "received_image.jpg"
    file.save(upload_path)

    try:
        image = face_recognition.load_image_file(upload_path)
        face_locations = face_recognition.face_locations(image)
        face_encodings = face_recognition.face_encodings(image, face_locations)

        if not face_encodings:
            return jsonify({"message": "No face detected", "access": "denied"}), 200

        for face_encoding in face_encodings:
            matches = face_recognition.compare_faces(known_encodings, face_encoding, tolerance=0.5)
            face_distances = face_recognition.face_distance(known_encodings, face_encoding)

            best_match_index = np.argmin(face_distances) if face_distances.size > 0 else -1
            if best_match_index != -1 and matches[best_match_index]:
                name = known_names[best_match_index]
                return jsonify({"message": f"Welcome {name}<3", "access": "granted"}), 200

        return jsonify({"message": "Unknown Face", "access": "denied"}), 200

    except Exception as e:
        return jsonify({"message": f"Error processing image: {str(e)}", "access": "denied"}), 500
    finally:
        if os.path.exists(upload_path):
            os.remove(upload_path)

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)
```

---

### **CODE 1: ESP32-CAM Arduino Code**

Code: Select all

#include <WiFi.h>
#include <HTTPClient.h>
#include "esp_camera.h"
#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SH110X.h>

// WiFi and Server Settings
#define SSID "wifiaddress"
#define PASSWORD "12345678910"
#define SERVER_URL "http://192.168.1.34:5000/upload"

// Camera Pin Definitions
#define PWDN_GPIO_NUM    32
#define RESET_GPIO_NUM   -1
#define XCLK_GPIO_NUM     0
#define SIOD_GPIO_NUM    26
#define SIOC_GPIO_NUM    27
#define Y9_GPIO_NUM      35
#define Y8_GPIO_NUM      34
#define Y7_GPIO_NUM      39
#define Y6_GPIO_NUM      36
#define Y5_GPIO_NUM      21
#define Y4_GPIO_NUM      19
#define Y3_GPIO_NUM      18
#define Y2_GPIO_NUM       5
#define VSYNC_GPIO_NUM   25
#define HREF_GPIO_NUM    23
#define PCLK_GPIO_NUM    22

// OLED Display Settings
#define SCREEN_WIDTH 128
#define SCREEN_HEIGHT 64
#define OLED_RESET    -1  
#define SDA_PIN 15
#define SCL_PIN 13

// Hardware Pins
#define BUZZER_PIN    2   // Buzzer
#define BUTTON_PIN    12  // Push button (doorbell)
#define IR_SENSOR_PIN 16  // IR motion sensor
#define LOCK_PIN      4   // Relay control pin for solenoid lock

Adafruit_SH1106G display(SCREEN_WIDTH, SCREEN_HEIGHT, &Wire, OLED_RESET);

void setupCamera() {
    camera_config_t config;
    config.ledc_channel = LEDC_CHANNEL_0;
    config.ledc_timer = LEDC_TIMER_0;
    config.pin_d0 = Y2_GPIO_NUM;
    config.pin_d1 = Y3_GPIO_NUM;
    config.pin_d2 = Y4_GPIO_NUM;
    config.pin_d3 = Y5_GPIO_NUM;
    config.pin_d4 = Y6_GPIO_NUM;
    config.pin_d5 = Y7_GPIO_NUM;
    config.pin_d6 = Y8_GPIO_NUM;
    config.pin_d7 = Y9_GPIO_NUM;
    config.pin_xclk = XCLK_GPIO_NUM;
    config.pin_pclk = PCLK_GPIO_NUM;
    config.pin_vsync = VSYNC_GPIO_NUM;
    config.pin_href = HREF_GPIO_NUM;
    config.pin_sscb_sda = SIOD_GPIO_NUM;
    config.pin_sscb_scl = SIOC_GPIO_NUM;
    config.pin_pwdn = PWDN_GPIO_NUM;
    config.pin_reset = RESET_GPIO_NUM;
    config.xclk_freq_hz = 20000000;
    config.pixel_format = PIXFORMAT_JPEG;
    config.frame_size = FRAMESIZE_QQVGA; // Smaller resolution (160x120)
    config.jpeg_quality = 12; // Higher quality
    config.fb_count = 1;

    if (esp_camera_init(&config) != ESP_OK) {
        Serial.println("Camera init failed! Restarting...");
        delay(5000);
        ESP.restart();
    }
    Serial.println("Camera initialized successfully");
}

void connectWiFi() {
    WiFi.begin(SSID, PASSWORD);
    Serial.print("Connecting to WiFi...");
    unsigned long wifiStartTime = millis();
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
        if (millis() - wifiStartTime > 15000) { // Timeout after 15 seconds
            Serial.println("\nWiFi connection failed. Restarting...");
            ESP.restart();
        }
    }
    Serial.println("\nConnected to WiFi!");
}

bool sendImageAndGetResponse() {
    camera_fb_t *fb = esp_camera_fb_get();
    if (!fb || fb->len == 0 || !fb->buf) {
        Serial.println("Camera capture failed!");
        if (fb) esp_camera_fb_return(fb);
        return false;
    }
    Serial.printf("Captured image size: %d bytes\n", fb->len);

    String boundary = "----ESP32Boundary";
    String bodyStart = "--" + boundary + "\r\n";
    bodyStart += "Content-Disposition: form-data; name=\"image\"; filename=\"image.jpg\"\r\n";
    bodyStart += "Content-Type: image/jpeg\r\n\r\n";
    String bodyEnd = "\r\n--" + boundary + "--\r\n";

    size_t payloadSize = bodyStart.length() + fb->len + bodyEnd.length();
    uint8_t *payload = (uint8_t *)malloc(payloadSize);
    if (!payload) {
        Serial.println("Failed to allocate memory for payload. Restarting...");
        esp_camera_fb_return(fb);
        delay(1000);
        ESP.restart();
    }

    memcpy(payload, bodyStart.c_str(), bodyStart.length());
    memcpy(payload + bodyStart.length(), fb->buf, fb->len);
    memcpy(payload + bodyStart.length() + fb->len, bodyEnd.c_str(), bodyEnd.length());

    HTTPClient http;
    bool accessGranted = false;

    http.begin(SERVER_URL);
    http.addHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
    http.addHeader("Content-Length", String(payloadSize));

    int httpResponseCode = http.POST(payload, payloadSize);
    if (httpResponseCode == 200) {
        String response = http.getString();
        Serial.printf("HTTP Response: %s\n", response.c_str());

        if (response.indexOf("\"access\": \"granted\"") != -1) {
            accessGranted = true;
        } else {
            accessGranted = false;
        }
    } else {
        Serial.printf("Error: HTTP Response Code %d\n", httpResponseCode);
        accessGranted = false; // Default to denied in case of error
    }

    http.end(); // Close the HTTP connection
    free(payload); // Free the allocated memory
    esp_camera_fb_return(fb); // Release the frame buffer
    return accessGranted;
}

void setup() {
    Serial.begin(115200);
    Wire.begin(SDA_PIN, SCL_PIN);

    // Initialize OLED Display
    if (!display.begin(0x3C, true)) { 
        Serial.println(F("SH1106 allocation failed"));
        for (;;); // Stop execution if the display fails
    }
    display.clearDisplay();
    display.setTextSize(2);
    display.setTextColor(SH110X_WHITE);
    resetOLED();

    pinMode(BUZZER_PIN, OUTPUT);
    pinMode(BUTTON_PIN, INPUT_PULLUP);
    pinMode(IR_SENSOR_PIN, INPUT);
    pinMode(LOCK_PIN, OUTPUT);  
    digitalWrite(BUZZER_PIN, LOW);
    digitalWrite(LOCK_PIN, LOW);

    setupCamera();
    connectWiFi();
    Serial.println("System Initialized.");
}

void loop() {
    int buttonState = digitalRead(BUTTON_PIN);
    int motionDetected = digitalRead(IR_SENSOR_PIN);

    Serial.print("Button state: ");
    Serial.println(buttonState == LOW ? "Pressed" : "Not Pressed");
    Serial.print("Motion detected: ");
    Serial.println(motionDetected == LOW ? "Yes" : "No");

    if (buttonState == LOW) {
        Serial.println("Doorbell Pressed!");
        showMessage("Doorbell  Ringing");
        delay(1000);
        tone(BUZZER_PIN, 1000);
        delay(500);
        noTone(BUZZER_PIN);
        delay(1000);
        resetOLED();
    }

    if (motionDetected == LOW || buttonState == LOW) {
        Serial.println("Trigger detected: Capturing image...");
        showMessage("Capturing Face...");
        delay(1000);

        // Send image and get server response
        bool accessGranted = sendImageAndGetResponse();

        if (accessGranted) {
            Serial.println("Face recognized: Access Granted.");
            showMessage("Access\nGranted\nWelcome!");
            digitalWrite(LOCK_PIN, HIGH);  // Unlock door
            delay(5000);  // Keep unlocked for 5 seconds
            digitalWrite(LOCK_PIN, LOW);   // Lock door again
            Serial.println("Unlock the door");
        } else {
            Serial.println("Unknown Face: Access Denied.");
            digitalWrite(LOCK_PIN, LOW); 
            Serial.println("Lock the door");
            showMessage("Access \nDenied!\nAlert!");
            delay(1000);
            for (int i = 0; i < 3; i++) {
                tone(BUZZER_PIN, 800);
                delay(150);
                tone(BUZZER_PIN, 1200);
                delay(150);
            }
            noTone(BUZZER_PIN);
        }

        delay(1500);
        resetOLED();
    }

    delay(500);
}

void showMessage(const char *message) {
    display.clearDisplay();
    display.setCursor(0, 10);
    display.setTextSize(2);
    display.setTextColor(SH110X_WHITE);  
    display.print(message);
    display.display();
}

void resetOLED() {
    showMessage("Welcome!\nSystem is Ready");
    delay(2000);
}

### **ERROR MESSAGE (ESP32-CAM Serial Monitor)**

Code: Select all

```
Connecting to WiFi...
Connected to WiFi!
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40094c91  PS      : 0x00060430  A0      : 0x800946bc  A1      : 0x3ffb1fc0  
A2      : 0xffffffff  A3      : 0x000002f4  A4      : 0x3ffb20b4  A5      : 0x00060423  
A6      : 0x3ffc7eb0  A7      : 0x00000000  A8      : 0x000002f4  A9      : 0xfffffffc  
A10     : 0x00000003  A11     : 0xffffffff  A12     : 0x00060420  A13     : 0x3f419ba0  
A14     : 0xffffffff  A15     : 0x00000000  SAR     : 0x0000001a  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x0000000f  LBEG    : 0x4008b9a0  LEND    : 0x4008b9ab  LCOUNT  : 0x00000000  
``

Any ideas what’s causing this crash right after Wi-Fi connects and the image is about to be sent? and how to solve it NOTE : Both the ESP32-CAM circuit and the Python Flask server were tested individually and work perfectly. The ESP32-CAM successfully captures images and controls hardware components such as the solenoid lock, buzzer, and OLED screen. Similarly, the Flask server accurately performs face recognition and returns the appropriate response ("access": "granted" or "access": "denied").
However, the issue arises when integrating them — that is, when the ESP32-CAM attempts to send the image to the Flask server and process the server's response to trigger the circuit. This part is not functioning correctly, even though both components are verified to work flawlessly on their own. Any help would be appreciated thank you

[Sprite]

You should have some kind of backtrace after that guru meditation command. If you can decode that, you can find what line caused the crash.

[nopnop2002]

Your logs don't show this.

Did it crash before you could take the photo?
Did it crash as soon as the loop task was executed?

Code: Select all

    Serial.println("System Initialized.");

    Serial.print("Button state: ");
    Serial.println(buttonState == LOW ? "Pressed" : "Not Pressed");
    Serial.print("Motion detected: ");
    Serial.println(motionDetected == LOW ? "Yes" : "No");