Esp32 allocate big two dimensional array in External memory

[Dario_Lobos]

I am doing a program for an Esp32, I am new in this. I did an array of two dimensions for storage the background of an small Display. I will use it to stream buffer all. Take some frames small where data change and using a font bit map replace uint16_t 565 pixel by bit points in the small frame. The rest after task send the array I will delete it.

I used a program to make the 565 one line hexa uint16_t

https://sourceforge.net/projects/lcd-im ... ter/files/

And I did a program to arrange it

https://drive.google.com/file/d/1vzO4_r ... sp=sharing

I try plenty ways to allocate memory for the two dimensional arrays and any works. I am asking information about it.

Code: Select all

#include <stdint.h>

#include "esp_heap_caps.h"

# define ARRAYBACKGROUNDSIZE (128*160)

uint16_t* pointerbackground[160][128]= (uint16_t*) heap_caps_malloc((ARRAYBACKGROUNDSIZE+100)*sizeof(uint16_t),MALLOC_CAP_SPIRAM);

static volatile uint16_t background [160][128]=

{{0xe75d, 0xf7ff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff,

0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xf7ff,

0xf7ff, 0xef9e, 0xdf3d, 0xe77d, 0xcefb, 0x9514, 0x3a4a, 0x31a8, 0x31a7, 0x39c8, 0x31a7, 0x2967, 0x2167, 0x2167, 0x2146, 0x2146,

0x2987, 0x2987, 0x2967, 0x2166, 0x1946, 0x2146, 0x2146, 0x2966, 0x39a6, 0x41c7, 0x41c7, 0x41e7, 0x49e7, 0x4a07, 0x4a07, 0x41e7,

0x41c7, 0x39a6, 0x3986, 0x3986, 0x3986, 0x3986, 0x3165, 0x3145, 0x3165, 0x41a6, 0x8308, 0xcc8b, 0xd4cb, 0xc44a, 0xdd0c, 0xed8d,

0xed8e, 0xed8d, 0xd4ec, 0xd4ec, 0xe54d, 0xdd0c, 0xd4ec, 0xdd4d, 0xedce, 0xedae, 0xed8e, 0xe56e, 0xe5b1, 0xe613, 0xeeb8, 0xef7d,

0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xf7ff, 0xf7ff,

0xf7ff, 0xffff, 0xffff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xf7ff, 0xdf5d},

and At the end or at start I am trying to reference one to the other and I can't.

Code: Select all

{ 0x82c7, 0x82e8, 0x7287, 0x7287, 0x51a5, 0x4984, 0x6a46, 0x8b08, 0x938b, 0x834b, 0x72c9, 0x6a05, 0x6a46, 0x7246, 0x7265, 0x6225,

0x6226, 0x72c9, 0xa42f, 0x836c, 0x834c, 0x7b2b, 0x7b6c, 0x7b6d, 0x736d, 0x736d, 0x7b8e, 0x5acb, 0x3187, 0x10e5, 0x0884, 0x0084,

0x00a4, 0x0906, 0x0907, 0x1148, 0x21a9, 0x41c8, 0x4985, 0x49a5, 0x6268, 0x6247, 0x72a9, 0x93cc, 0x6247, 0x5a05, 0x6226, 0x8309,

0x82c8, 0x7287, 0x82a7, 0x9348, 0xb42c, 0xbc6c, 0x9b68, 0x7a66, 0x6a05, 0x6a05, 0x7246, 0x59e5, 0x7aa7, 0x9329, 0xbc4c, 0xbc6d,

0x82c7, 0x49a4, 0x49e6, 0x6288, 0x9b8b, 0x8b8c, 0x7aea, 0x7288, 0x7288, 0x72a8, 0x7267, 0x7288, 0x6a47, 0x6226, 0x7288, 0x9329,

0x8b29, 0x8b29, 0x934a, 0x7267, 0x8ae9, 0x7246, 0x6a46, 0x7a86, 0x8a86, 0x92e8, 0x9329, 0x7aa7, 0x7266, 0x8ac7, 0x8b29, 0x8b08,

0x7246, 0x7226, 0x82a7, 0x8b07, 0x82c8, 0x7aa7, 0x7287, 0x59c5, 0x51a4, 0x6a45, 0x82e7, 0x82e8, 0x8ae8, 0x82c7, 0x6a45, 0x59e4,

0x51a4, 0x51a4, 0x72a7, 0x6a67, 0x4983, 0x5183, 0x61e4, 0x9348, 0xb40b, 0x9b69, 0x82a6, 0x9307, 0x7a65, 0x61c3, 0x5183, 0x7266}};

*pointerBackground =&background;

If anyone can help me is very welcome.

[MicroController]

Code: Select all

uint16_t* pointerbackground[160][128]

This declares a two-dimensional array of pointers; not what you want.

The cleanest way to realize what you want is probably to define a struct with your array inside it:

Code: Select all

typedef struct pixbuf {
  uint16_t pixels[160][128];
} pixbuf_t;

...

pixbuf_t* pb = (pixbuf_t*) heap_caps_malloc( sizeof(pixbuf_t),... );

The other, more fragile, solution would be to create a 2D array type:

Code: Select all

typedef uint16_t pixels_t[160][128];

...

pixels_t* pb = (pixels_t*) heap_caps_malloc( sizeof(pixels_t),... );

(*pb)[12][34] = 0x1234;

And a final option:

Code: Select all

uint16_t (*pointerbackground)[160][128] = ...

(Note the parentheses!)

Initializing a 2D array would be like

Code: Select all

static const pixels_t PIXELS = { 
  {1,2,3,...,128},
  {128,127,...,1},
  ...
};

(A 2D array in C is an array of 1D arrays.)

(Also mind your use of volatile.)

[Dario_Lobos]

Many thanks for your help. I found another solution searching on the web, that is do an array of pointers (see background.c int the below github link). I am making my first program. You can see the solution.

The program is this

https://github.com/DarioLobos/Battery-C ... /tree/main

I have another question I am passing in task a pointer of an structure and then moving in using a pointer to receive it and and sizeof(data type) to move up into it. Is it possible or is a mistake? I had the idea an I am not sure If can work. You can see in main the task and a pointer to time_def_t timers_config.timers[0] in which I use the sizeof to pointer to the array and then the moving to the next using the size of the array type. I don't know if exist other alternative.

Thanks for your comments.

[Dario_Lobos]

The circuit that control the es32 is this

https://drive.google.com/file/d/1SEHRKZ ... drive_link

[MicroController]

Code: Select all

void display_allocation(void){
...
  uint16_t background [160][128]=

  {{0xe75d, 0xf7ff, 0xffff, 
...

}

Don't do it like this. This will likely try and allocate the whole array on the stack for no reason. Either make your init data a (global) static const (will be put in flash and only there), or embed the data (also uses flash only).

You can pass any pointer to a task via its void* argument, then cast it back to the correct type inside the task; that's a valid pattern (as long as the pointer remains valid while the task uses it). Tasks can also access global variables like any other function, so passing a pointer to a global variable is often not really necessary. (The task argument is most useful when you have the same task code running concurrently with more than one instance, in which case the argument is the way to tell each task instance what exactly it is to do.)

[MicroController]

I found another solution searching on the web, that is do an array of pointers

Note that this is not the same as a 2D array. It may look like it at first because you can write arr[x][y], but it does not allocate all the memory in one continuous block. The latter is usually desirable/required so that you can process (send to a display...) all data in one go instead of having to do it one line at a time.

[Dario_Lobos]

I did this is the start up of the program then all is deleted

Code: Select all

void display_allocation(void){

// Allocate memory for each row in PSRAM
for (int i = 0; i < ROWARRAY; i++) {
    background_pointers[i] = (uint16_t*)heap_caps_malloc(COLARRAY * sizeof(uint16_t), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
    if (background_pointers[i] == NULL) {
        printf("Failed to allocate row %d\n", i);
        return;
        }
}

// after transfer to EXT RAM clean this array SEE BELLOW 

uint16_t background [COLARRAY][ROWARRAY]=

{{0xe75d, 0xf7ff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff,
 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xffff, 0xf7ff,

//.....(alll the background)

for (int i = 0; i < ROWARRAY; i++) {
	for (int j = 0; j < COLARRAY; j++) {
	
background_pointers[i][j] = background [i][j];
background [i][j]=0;

static void display_init(void *pvparameter){
spi_device_handle_t spi= pvparameter;
display_allocation();
setup_time_bkg_allocation();

xTaskNotifyGive(xtaskHandleFrame);

uint8_t array_of_commands_poll[11]={ NORON,COLMOD,PCOLMOD,DISPON,CASET,0,COLARRAY-1,RASET,0,ROWARRAY-1,RAMWR};

spi_polling(spi, *pointer_to_commands_poll,sizeof(array_of_commands_poll), true);

spi_polling(spi, background_pointers[0][0],sizeof(background_pointers), true);// ALL THE SCREEN

spi_device_polling_end(spi, portMAX_DELAY);

// BACKGROUND READY AT FIRST

// free memory and delete task awaiting next task finish


ulTaskNotifyTake(pdTRUE,portMAX_DELAY);

free(background_pointers);

  vTaskDelete(NULL);
}

static void display_update (void *pvparameter){

int received_voltage;
 
int digits;

uint8_t received_digit;


// Allocate memory for each row in PSRAM
for (int i = 0; i < ROWAC; i++) {
    ac_pointers_to_send[i] = (uint16_t*)heap_caps_malloc(COLAC * sizeof(uint16_t), MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
    if (ac_pointers_to_send[i] == NULL) {
        printf("Failed to allocate AC row %d\n", i);
        return;
        }
}


for(;;){

ulTaskNotifyTake(pdTRUE, portMAX_DELAY);

digits=-1;

received_voltage= (int) pvparameter; 

received_voltage= (int)((received_voltage*110000/2350)+5)/10; //transform to AC , eliminate one digit rounding, 

if ((received_digit=received_voltage-received_voltage%10000/10000)>0){

digits=0;

	for (int j=0;j<8;j++){
		
		for(int i=0;i<8;i++){
			if((font_bits[received_digit][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}

}

else if((received_digit=((received_voltage%10000-received_voltage%1000)/1000)>0) | (digits >-1)){
digits++; 
	for (int j=digits*8;j<(digits*8+8);j++){
		for(int i=0;i<8;i++){
			if((font_bits[received_digit][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}


}
else if((received_digit=((received_voltage%1000-received_voltage%100)/100)>0) | (digits >-1)){
digits++; 

	for (int j=digits;j<(digits*8+8);j++){
		for(int i=0;i<8;i++){
			if((font_bits[received_digit][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}
}
else if((received_digit=((received_voltage%100-received_voltage%10)/10)>0) | (digits >-1)){
digits++; 

	for (int j=digits;j<(digits*8+8);j++){
		for(int i=0;i<8;i++){
			if((font_bits[received_digit][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}
}
else {
received_digit=(received_voltage-received_voltage)%10;

digits++; 

	for (int j=digits;j<(digits*8+8);j++){
		for(int i=0;i<8;i++){
			if((font_bits[received_digit][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}
}
	for (int j=24;j<32;j++){
		for(int i=0;i<8;i++){
			if((font_bits[11][j]&(1<<i))>0){
				ac_pointers_to_send[i][j]=ACCOLOR;
				}
			else{
				ac_pointers_to_send[i][j]=ac_pointers[i][j];

				}
		}
	}
spi_transmit_isr(spi,true,*pointer_to_commands_isr_ac, sizeof(array_of_commands_ISR_AC), true);

spi_transmit_isr(spi,false, ac_pointers_to_send[0][0], sizeof(ac_pointers_to_send), true);
 

}
}


void spi_polling(spi_device_handle_t spi, const uint8_t data,const int lenght,  bool keep_cs_active)
{
    esp_err_t ret;
    spi_transaction_t t;
    memset(&t, 0, sizeof(t));       //Zero out the transaction
    t.length = lenght;                   
    t.tx_buffer = &data;             //The data is the command and next polling parameter,s
    t.user = (void*)0;              //D/C needs to be set to 0
    if (keep_cs_active) {
        t.flags = SPI_TRANS_CS_KEEP_ACTIVE;   //Keep CS active after data transfer
    }
    ret = spi_device_polling_transmit(spi, &t); //Transmit!
    assert(ret == ESP_OK);          //Should have had no issues.
}



void spi_transmit_isr(spi_device_handle_t spi,bool command, const uint8_t data,const int lenght,  bool keep_cs_active) {

    esp_err_t ret;
    spi_transaction_t t;
    memset(&t, 0, sizeof(t));       //Zero out the transaction
    t.length = lenght;                   
    t.tx_buffer = &data;             //The the command as data and data  next transmit parameters
    t.user = (void*)0;              //D/C needs to be set to 0
    if (keep_cs_active) {
        t.flags = SPI_TRANS_CS_KEEP_ACTIVE;   //Keep CS active after data transfer
    }

	if (command){
    	ret = spi_device_queue_trans(spi, &t, portMAX_DELAY); //Transmit!
    	assert(ret == ESP_OK);          //Should have had no issues.

	}
	else{
    	ret = spi_device_queue_trans(spi, &t, portMAX_DELAY); //Transmit!
    	assert(ret == ESP_OK);          //Should have had no issues.

		spi_transaction_t *tresult=&t; // not used just to don't free the spi and unblock task to next send.


    	spi_device_get_trans_result(spi,&tresult, portMAX_DELAY);
	}
	}

if you want to see the code complete I am updating it, If you have any comment is welcome.

https://github.com/DarioLobos/Battery-C ... rter-ESP32

After upload keep only the parts that change and the rest I erase it. In there just change adding numbers (volts and time) and some banners. I will use an IC2 keypad and a RTC clock to set up time and actions like charge with the public electric line if some time is reached and battery charge is not enough, or free gpio to start lamps or water plants.
Are my first programs with so much time or readings about it.

Thanks for your help. I will read your comments carefully. Your web site information is quite complete Many thank!!

Thanks