ESP32 heap management and the esp_lcd_new_i80_bus API

FrankNabas
Posts: 6
Joined: Wed Jun 18, 2025 12:44 am

ESP32 heap management and the esp_lcd_new_i80_bus API

Postby FrankNabas » Tue Jun 24, 2025 11:35 pm

Greetings everyone!

I am trying to use a LCD TFT display using the esp-idf environment and I stumbled across a problem that I don't know how to solve.
The example I'm following is the i80_controller_example_main.c (https://github.com/espressif/esp-idf/bl ... ple_main.c)

When flashing the code for the first time I got an out-of-memory error from esp_lcd_new_i80_bus().
I found that strange since my code does nothing before calling this function, that is, no heap memory allocations.
Investigating the function I saw that it also doesn't perform any significant allocation before the one that fails.
This is the snippet where it tries to allocate:

Code: Select all

esp_err_t ret = ESP_OK;
    esp_lcd_i80_bus_t *bus = NULL;
    ESP_RETURN_ON_FALSE(bus_config && ret_bus, ESP_ERR_INVALID_ARG, TAG, "invalid argument");
    // although LCD_CAM can support up to 24 data lines, we restrict users to only use 8 or 16 bit width
    ESP_RETURN_ON_FALSE(bus_config->bus_width == 8 || bus_config->bus_width == 16, ESP_ERR_INVALID_ARG,
                        TAG, "invalid bus width:%d", bus_config->bus_width);
    // allocate i80 bus memory <~ This is the call that fails.
    bus = heap_caps_calloc(1, sizeof(esp_lcd_i80_bus_t), LCD_I80_MEM_ALLOC_CAPS);
    ESP_GOTO_ON_FALSE(bus, ESP_ERR_NO_MEM, err, TAG, "no mem for i80 bus");
    bus->bus_width = bus_config->bus_width;
    bus->bus_id = -1;
I then set up a heap debugging code to see if it was indeed asking for more memory that's available. This is the code:

Code: Select all

// Heap debugging.
// We get extra information about allocations on the heap by overloading the functions 'esp_heap_trace_alloc_hook'
// and 'esp_heap_trace_free_hook'. It's very limited what you can do with these functions because they will get called
// right after the heaps are created and before most of the underlying system initializing. So writing to the console from
// them is not possible until we get to 'app_main'.
// To circumvent this we created function pointer types where we assign two different functions, one to be called before 'app_main'
// and one to be called after. The one called after simply uses 'ESP_LOGW'. The one before stores the allocation and free information
// in limited size arrays, so we can print the info later on 'app_main'. I'm sure this is not the best way to do this, but it works, and
// I tried a lot of other things without success.

/**
 * @brief A structure representing a heap memory allocation.
 * 
 * @property ptr The pointer returned by the allocation API.
 * @property size The size of the requested buffer.
 * @property caps The 'MALLOC_CAP_*' flags.
 */
struct heap_alloc_info_t {
    void* ptr;
    size_t size;
    uint32_t caps;
};

// The number of allocations we stored in the buffer.
static uint8_t master_heap_alloc_count = 0;

// The number of frees we stored in the buffer.
static uint8_t master_heap_free_count = 0;

// The buffer to hold allocation information.
static heap_alloc_info_t master_heap_alloc_info[20];

// The buffer to hold free information.
static void* master_heap_free_info[20];

/**
 * @brief Function pointer definition following the signature of 'esp_heap_trace_alloc_hook'.
 * 
 * @param ptr The pointer returned by then allocation API.
 * @param size The size of the requested buffer.
 * @param caps The 'MALLOC_CAP_*' flags.
 */
typedef void (*heap_trace_alloc_hook_t)(
    void* ptr,
    size_t size,
    uint32_t caps
);

/**
 * @brief Function pointer definition followinf the signature of 'esp_heap_trace_free_hook'.
 * 
 * @param ptr The pointer of the memory being freed.
 */
typedef void (*heap_trace_free_hook_t)(void* ptr);

// Tag for 'ESP_LOG*'.
static const char* HEAP_DEBUG_TAG = "Heap Debugging";

/**
 * @brief This is the function that gets called by 'esp_heap_trace_alloc_hook' before 'app_main' is called.
 *        It stores the allocations information in the buffer to be printed later on 'app_main'.
 * 
 * @param ptr The pointer returned by the allocation API.
 * @param size The size of the requested buffer.
 * @param caps The 'MALLOC_CAP_*' flags.
 */
void esp_heap_trace_alloc_hook_primitive(void* ptr, size_t size, uint32_t caps) {
    // If it's the first call zero out the array.
    if (master_heap_alloc_count == 0)
        memset(master_heap_alloc_info, 0, sizeof(master_heap_alloc_info));

    // The number of allocation information we can store is limited by the array size.
    // So we check if we passed that to avoid access violation.
    if (master_heap_alloc_count >= sizeof(master_heap_alloc_info))
        return;

    // Storing allocation information.
    master_heap_alloc_info[master_heap_alloc_count++] = {
        .ptr = ptr,
        .size = size,
        .caps = caps,
    };
}

/**
 * @brief This is the function that gets called by 'esp_heap_trace_alloc_hook' after 'app_main' is called.
 *        It simply prints the allocation information to the console.
 *        Its parameters are the same as the previous function.
 */
void esp_heap_trace_alloc_hook_log(void* ptr, size_t size, uint32_t caps) {
    ESP_LOGW(HEAP_DEBUG_TAG, "Heap allocation. Addr: %p; Size: %u; Caps: %lu;", ptr, size, caps);
}

/**
 * @brief This is the function that gets called by 'esp_heap_trace_free_hook' before 'app_main' is called.
 *        It's identical to the 'alloc' one, except this one only stores the pointer of the memory being freed.
 * 
 * @param ptr The pointer of the memory being freed.
 */
void esp_heap_trace_free_hook_primitive(void* ptr) {
    if (master_heap_free_count == 0)
        memset(master_heap_free_info, 0, sizeof(master_heap_free_info));

    if (master_heap_free_count >= sizeof(master_heap_free_info))
        return;

    master_heap_free_info[master_heap_free_count++] = ptr;
}

/**
 * @brief This is the function that gets called by 'esp_heap_trace_free_hook' after 'app_main' is called.
 *        It simply prints the freed memory address to the console.
 *        Its parameters are the same as the previous function.
 */
void esp_heap_trace_free_hook_log(void* ptr) {
    ESP_LOGW(HEAP_DEBUG_TAG, "Heap free. Addr: %p;", ptr);
}

// Setting our function pointers to the primitive functions, I.E., the ones called before 'app_main' is called.
static heap_trace_alloc_hook_t alloc_hook = esp_heap_trace_alloc_hook_primitive;
static heap_trace_free_hook_t free_hook = esp_heap_trace_free_hook_primitive;

// These are the actual overloads that will be called from the 'heap_caps*' APIs.
// These functions are originally defined with '__attribute__((weak))'
// The parameters are the same as our previous functions and they only call our function pointers.
void esp_heap_trace_alloc_hook(void* ptr, size_t size, uint32_t caps) {
    alloc_hook(ptr, size, caps);
}

void esp_heap_trace_free_hook(void* ptr) {
    free_hook(ptr);
}

/**
 * @brief This function is the callback that will be called when a heap allocation fails.
 *        There is no problem with this function being called before 'app_main' because is there where we
 *        assign this callback with 'heap_caps_register_failed_alloc_callback'.
 * 
 * @param requested_size The allocation requested size.
 * @param caps The 'MALLOC_CAP_*' flags.
 * @param function_name The name of the function attempting the allocation.
 */
void heap_caps_alloc_failed_hook(size_t requested_size, uint32_t caps, const char* function_name) {
    ESP_LOGE(HEAP_DEBUG_TAG, "Failed to allocate. Size: %u; Caps: %lu; Function: %s;", requested_size, caps, function_name);
}

extern "C" void app_main(void) {
    ESP_LOGW(HEAP_DEBUG_TAG, "'app_main()' begin;");

    ESP_LOGW(HEAP_DEBUG_TAG, "Printing previous allocations;");
    for (uint8_t i = 0; i < master_heap_alloc_count; i++) {
        auto current_allocation = master_heap_alloc_info[i];
        if (current_allocation.ptr)
            ESP_LOGW(HEAP_DEBUG_TAG, "Heap allocation. Addr: %p; Size: %u; Caps: %lu;", current_allocation.ptr, current_allocation.size, current_allocation.caps);
    }

    ESP_LOGW(HEAP_DEBUG_TAG, "Printing previous frees;");
    for (uint8_t i = 0; i < master_heap_free_count; i++) {
        auto current_free_address = master_heap_free_info[i];
        if (current_free_address)
            ESP_LOGW(HEAP_DEBUG_TAG, "Heap free. Addr: %p;", current_free_address);
    }

    // Replacing the 'primitive' hooks with the logging ones.
    alloc_hook = esp_heap_trace_alloc_hook_log;
    free_hook = esp_heap_trace_free_hook_log;
    
    // Registering the allocation failure callback.
    ESP_ERROR_CHECK(heap_caps_register_failed_alloc_callback(heap_caps_alloc_failed_hook));

    // Printing heap information to compare with the allocations being requested.
    multi_heap_info_t default_caps_heap_info{ };
    heap_caps_get_info(&default_caps_heap_info, MALLOC_CAP_DEFAULT);
    ESP_LOGW(HEAP_DEBUG_TAG, "First heap information gather;");
    ESP_LOGW(HEAP_DEBUG_TAG, "Default CAP heap info:\n\tAllocated blocks: %u;\n\tFree blocks: %u;\n\tLargest free block: %u;\n\tMin free bytes: %u;\n\tTotal allocated bytes: %u;\n\tTotal blocks: %u;\n\tTotal free bytes: %u;",
        default_caps_heap_info.allocated_blocks, default_caps_heap_info.free_blocks, default_caps_heap_info.largest_free_block, default_caps_heap_info.minimum_free_bytes, default_caps_heap_info.total_allocated_bytes,
        default_caps_heap_info.total_blocks, default_caps_heap_info.total_free_bytes);

    
    // Calling the function that will cause the allocation failure.
    ESP_LOGW(HEAP_DEBUG_TAG, "Calling: i80_tft_lcd_init();");
    i80_tft_lcd_init();
    
    \/ \/ \/ Rest of app_main code \/ \/ \/
The results I got back were interesting, here's the monitor console output:

Image
Image

So indeed, the function is asking for a lot of memory, more than what's available in a single block.

My questions are:
1- Is this total free memory amount normal? 249756 bytes is less than the half of the 520kb available. I'm fairly new to ESP32, and I honestly couldn't find information about it.
My chip information:

Chip is ESP32-D0WD-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
Crystal is 40MHz


I was also worried it might have been fragmentation, but the largest block available have 118784.

2- Is this library behavior normal? Or it's something I'm doing wrong that's causing it to request an enormous (128004B) amount of memory?

In the allocation call it requests the 'sizeof(esp_lcd_i80_bus_t)':
bus = heap_caps_calloc(1, sizeof(esp_lcd_i80_bus_t), LCD_I80_MEM_ALLOC_CAPS);
I added a logging entry just before the allocation attempt to print 'sizeof(esp_lcd_i80_bus_t)' and I got 68 bytes.
Is it trying to get more because it's calling 'heap_caps_calloc' with one contiguous block of memory?
heap_caps_calloc definition:

Code: Select all

/**
 * @brief Allocate a chunk of memory which has the given capabilities. The initialized value in the memory is set to zero.
 *
 * Equivalent semantics to libc calloc(), for capability-aware memory.
 *
 * In IDF, ``calloc(p)`` is equivalent to ``heap_caps_calloc(p, MALLOC_CAP_8BIT)``.
 *
 * @param n    Number of continuing chunks of memory to allocate
 * @param size Size, in bytes, of a chunk of memory to allocate
 * @param caps        Bitwise OR of MALLOC_CAP_* flags indicating the type
 *                    of memory to be returned
 *
 * @return A pointer to the memory allocated on success, NULL on failure
 */
void *heap_caps_calloc(size_t n, size_t size, uint32_t caps);
*EDIT* I tried allocating memory myself with both 'heap_caps_malloc' and 'heap_caps_calloc' and everything worked out:

Image

Code: Select all

// Random allocation test.
ESP_LOGW(HEAP_DEBUG_TAG, "Attempting to allocate 68 bytes with malloc.");
void* test_buffer1 = heap_caps_malloc(68, MALLOC_CAP_DEFAULT);

ESP_LOGW(HEAP_DEBUG_TAG, "Attempting to allocate 68 bytes with calloc.");
void* test_buffer2 = heap_caps_calloc(1, 68, MALLOC_CAP_DEFAULT);

heap_caps_free(test_buffer1);
heap_caps_free(test_buffer2);
I'll post the code I took from the example, I'm probably doing something wrong.

Thanks a lot!

Code: Select all

#pragma once

#include <esp_lcd_io_i80.h>
#include <esp_lcd_panel_io.h>
#include <esp_lcd_panel_vendor.h>
#include <esp_lcd_panel_ops.h>

#include <lvgl.h>

#define I80_LCD_RST_GPIO             15
#define I80_LCD_CS_GPIO              2
#define I80_LCD_RS_GPIO              4
#define I80_LCD_WR_GPIO              5
#define I80_LCD_RD_GPIO              18

#define I80_LCD_DATA_0               13
#define I80_LCD_DATA_1               12
#define I80_LCD_DATA_2               14
#define I80_LCD_DATA_3               27
#define I80_LCD_DATA_4               26
#define I80_LCD_DATA_5               25
#define I80_LCD_DATA_6               33
#define I80_LCD_DATA_7               32

#define I80_LCD_DATA_GPIO {          \
    I80_LCD_DATA_0,                  \
    I80_LCD_DATA_1,                  \
    I80_LCD_DATA_2,                  \
    I80_LCD_DATA_3,                  \
    I80_LCD_DATA_4,                  \
    I80_LCD_DATA_5,                  \
    I80_LCD_DATA_6,                  \
    I80_LCD_DATA_7,                  \
}

#define I80_LCD_PIXEL_CLK_HZ         10000000
#define I80_LCD_H_RES                320
#define I80_LCD_V_RES                240
#define I80_LCD_CMD_BITS             8
#define I80_LCD_PARAM_BITS           8
#define I80_LCD_BUS_WIDTH            8
#define I80_LCD_DMA_BURST_SIZE       64

#define I80_LCD_LVGL_DRAW_BUF_LINES  100

static bool i80_tft_lcd_notify_flush_ready(esp_lcd_panel_io_handle_t io_handle, esp_lcd_panel_io_event_data_t* event_data, void* user_data) {
    lv_display_flush_ready(reinterpret_cast<lv_display_t*>(user_data));
    return false;
}

static void i80_tft_lcd_flush_cb(lv_display_t* display, const lv_area_t* area, uint8_t* color_map) {
    esp_lcd_panel_handle_t panel_handle = reinterpret_cast<esp_lcd_panel_handle_t>(lv_display_get_user_data(display));
    int offsetx1 = area->x1;
    int offsetx2 = area->x2;
    int offsety1 = area->y1;
    int offsety2 = area->y2;

    // because LCD is big-endian, we need to swap the RGB bytes order
    lv_draw_sw_rgb565_swap(color_map, (offsetx2 + 1 - offsetx1) * (offsety2 + 1 - offsety1));
    // copy a buffer's content to a specific area of the display
    esp_lcd_panel_draw_bitmap(panel_handle, offsetx1, offsety1, offsetx2 + 1, offsety2 + 1, color_map);
}

void i80_tft_lcd_init() {
    esp_lcd_i80_bus_handle_t bus_handle;
    esp_lcd_panel_io_handle_t io_handle;
    esp_lcd_panel_handle_t panel_handle;

    // Bus init.
    esp_lcd_i80_bus_config_t bus_config{
        .dc_gpio_num         = I80_LCD_RS_GPIO,
        .wr_gpio_num         = I80_LCD_WR_GPIO,
        .clk_src             = LCD_CLK_SRC_DEFAULT,
        .data_gpio_nums      = I80_LCD_DATA_GPIO,
        .bus_width           = I80_LCD_BUS_WIDTH,
        .max_transfer_bytes  = I80_LCD_H_RES * 100 * sizeof(uint16_t),
        .dma_burst_size      = I80_LCD_DMA_BURST_SIZE,
    };

    ESP_ERROR_CHECK(esp_lcd_new_i80_bus(&bus_config, &bus_handle));

    esp_lcd_panel_io_i80_config_t io_config{
        .cs_gpio_num         = I80_LCD_CS_GPIO,
        .pclk_hz             = I80_LCD_PIXEL_CLK_HZ,
        .trans_queue_depth   = 10,
        .lcd_cmd_bits        = I80_LCD_CMD_BITS,
        .lcd_param_bits      = I80_LCD_PARAM_BITS,
        .dc_levels           = {
            .dc_idle_level   = 0,
            .dc_cmd_level    = 0,
            .dc_dummy_level  = 0,
            .dc_data_level   = 1,
        },
    };

    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i80(bus_handle, &io_config, &io_handle));

    // Panel init.
    esp_lcd_panel_dev_config_t panel_config{
        .reset_gpio_num  = I80_LCD_RST_GPIO,
        .rgb_ele_order   = LCD_RGB_ELEMENT_ORDER_RGB,
        .bits_per_pixel  = 16,
    };

    ESP_ERROR_CHECK(esp_lcd_new_panel_st7789(io_handle, &panel_config, &panel_handle));

    esp_lcd_panel_reset(panel_handle);
    esp_lcd_panel_init(panel_handle);
    esp_lcd_panel_invert_color(panel_handle, true);
    esp_lcd_panel_set_gap(panel_handle, 0, 20);

    lv_init();
    lv_display_t *display = lv_display_create(I80_LCD_H_RES, I80_LCD_V_RES);

    size_t draw_buffer_size = I80_LCD_H_RES * I80_LCD_LVGL_DRAW_BUF_LINES * sizeof(lv_color16_t);
    void *buffer1 = esp_lcd_i80_alloc_draw_buffer(io_handle, draw_buffer_size, 0);
    void *buffer2 = esp_lcd_i80_alloc_draw_buffer(io_handle, draw_buffer_size, 0);
    assert(buffer1);
    assert(buffer2);

    lv_display_set_buffers(display, buffer1, buffer2, draw_buffer_size, LV_DISPLAY_RENDER_MODE_PARTIAL);
    lv_display_set_user_data(display, panel_handle);
    lv_display_set_color_format(display, LV_COLOR_FORMAT_RGB565);
    lv_display_set_flush_cb(display, i80_tft_lcd_flush_cb);

    lv_disp_set_default(display);
}

FrankNabas
Posts: 6
Joined: Wed Jun 18, 2025 12:44 am

Re: ESP32 heap management and the esp_lcd_new_i80_bus API

Postby FrankNabas » Fri Jun 27, 2025 1:59 am

Alright I've figured out. Will post the findings here if someone has the same problem.

Firstly I was wrong. The allocation attempt causing the problem wasn't the one allocating for the bus handle, but the one allocating for the format buffer. Here's the call:

Code: Select all

#if SOC_I2S_TRANS_SIZE_ALIGN_WORD
    // transform format for LCD commands, parameters and color data, so we need a big buffer
    bus->format_buffer = heap_caps_calloc(1, max_transfer_bytes, MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT | MALLOC_CAP_DMA);
#else
As we can see it's trying to allocate 'max_transfer_bytes'. If we check the code above we can find where it works with this value:

Code: Select all

size_t max_transfer_bytes = (bus_config->max_transfer_bytes + 3) & ~0x03; // align up to 4 bytes
#if SOC_I2S_TRANS_SIZE_ALIGN_WORD
    // double the size of the internal DMA buffer if bus_width is 8,
    // because one I2S FIFO (4 bytes) will only contain two bytes of valid data
    max_transfer_bytes = max_transfer_bytes * 16 / bus_config->bus_width + 4;
#endif
So it gets the max transfer bytes from the config we passed to it, aligns to 4 bytes and then determines the transfer block size by relating it to the bus width. This buffer size is the size of a single transaction.
Now looking into the example from the ESP-IDF repo, the max transfer bytes passed is calculated as follows:

Code: Select all

.max_transfer_bytes = EXAMPLE_LCD_H_RES * 100 * sizeof(uint16_t)
In my case my display horizontal resolution is 320 pixels.
320 * 100 * 2(size of uint16>t) = 64000.
We don't need to worry about the 4-byte alignment because this value is already aligned.

Now calculating the format buffer size.
On the example the bus width is set to 8.
64000 * 16 / 8 + 4 = 128004.
And look at that, the exact value was being requested.

So to work with more manageable buffer sizes we have to either increase the bus_width, decrease the max_transfer_bytes or both.
According to the documentation the bus width can be either 8 or 16 bits, so we choose 16.
The example they transfer 100 lines of pixels at most per transaction. Let's choose something more manageable, like 50 lines.

with the new values we have:
config.bus_width = 16
config.max_transfer_bytes = 32000

Applying the calculations (again we don't need to worry about the alignment because 32000 is already aligned):
32000 * 16 / 16 + 4 = 32004 (~31.25Kb)
Now that's a way more manageable buffer size.

Changing these values and compiling the program again we have success:

Image

And that's it! It's way better to figure out you configure something wrong than finding out the API is broken :D

Thanks!

Who is online

Users browsing this forum: Amazon [Bot], Bing [Bot] and 2 guests