pvPortMallocCaps with MALLOC_CAP_32BI and memset

commarmi76
Posts: 7
Joined: Sun Apr 23, 2017 10:06 pm

pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby commarmi76 » Sun Apr 23, 2017 10:17 pm

Hello,

I ported lame (libmp3lame) mp3 encode to esp32. I'm using pvPortMallocCaps with MALLOC_CAP_32BIT to get memory for two big int arrays. I wonder if I can use memset to initialize this arrays because according the documentation, memory allocated with MALLOC_CAP_32BIT needs to be accessed in 32bits words.
Is there someone that knows the answer ?

PD: The port is in https://github.com/commarmi76/libmp3lame_for_esp32 and apparently works :)

Thanks

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby ESP_Angus » Sun Apr 23, 2017 11:51 pm

Very exciting port!

Memset (and the similar newlib functions) should use only 32-bit word operations (for performance) as long as the argument buffer is at a 32-bit aligned address & is a 32-bit multiple of length. So if you're memsetting the entire buffer, this will be safe.

You can read the source here: https://github.com/espressif/newlib-esp ... g/memset.c

Out of interest, why are you using MALLOC_CAP_32BITS? Do you need to allocate from IRAM to avoid running out of DRAM?

commarmi76
Posts: 7
Joined: Sun Apr 23, 2017 10:06 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby commarmi76 » Mon Apr 24, 2017 4:41 am

@ESP_Angus:
You are right, I use MALLOC_CAP_32BITS because Lame creates a lot of buffers but two at the begining are specially big (48k echa one). Fortunately these are int buffers. You can see memory info at the start of the lame_test.c:

ESP32 SDK version:v2.0-rc1-590-g0ea4c3c0, RAM left 180060
get_lame_version(): 3.99.5
pre lame_init() free mem8bit: 179892 mem32bit: 284324
post lame_init() free mem8bit: 91724 mem32bit: 196156
....
post lame_init() free mem8bit: 21136 mem32bit: 29552



As you can see lame is a memory eater.

Br.

commarmi76
Posts: 7
Joined: Sun Apr 23, 2017 10:06 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby commarmi76 » Mon Apr 24, 2017 4:59 am

ESP_Angus wrote:Very exciting port!

Memset (and the similar newlib functions) should use only 32-bit word operations (for performance) as long as the argument buffer is at a 32-bit aligned address & is a 32-bit multiple of length. So if you're memsetting the entire buffer, this will be safe.

You can read the source here: https://github.com/espressif/newlib-esp ... g/memset.c

Out of interest, why are you using MALLOC_CAP_32BITS? Do you need to allocate from IRAM to avoid running out of DRAM?
other two questions:

- Can I use MALLOC_CAP_32BITS to create array of float ? I'm not sure if floats are accesed only in 32 bits way.

- When I try to create an array (float [64][64]) ) but I have no enough memory, the system don't crash but when I access to the array I write other parts of the memory. Maybe I have something wrong configured, did someone experienced something similary ?

thks.

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby ESP_Angus » Thu Apr 27, 2017 11:47 pm

commarmi76 wrote: - Can I use MALLOC_CAP_32BITS to create array of float ? I'm not sure if floats are accesed only in 32 bits way.
I'm fairly sure this will work.
commarmi76 wrote: - When I try to create an array (float [64][64]) ) but I have no enough memory, the system don't crash but when I access to the array I write other parts of the memory. Maybe I have something wrong configured, did someone experienced something similary ?
How are you creating this array, is it on the stack or in static memory or via malloc? Can you post a code snippet?

commarmi76
Posts: 7
Joined: Sun Apr 23, 2017 10:06 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby commarmi76 » Fri Apr 28, 2017 8:18 am

Hello,

you can see here: https://github.com/commarmi76/libmp3lam ... el.c#L1820
If I don't set CONFIG_MAIN_TASK_STACK_SIZE with enough memory this line don't crash but when the code reaches the memset below the array declaration, it overwrite (with 0) a global array defined here: https://github.com/commarmi76/libmp3lam ... pvt.c#L172
It sounds crazy, but it happens.

Thanks

f.h-f.s.
Posts: 214
Joined: Thu Dec 08, 2016 2:53 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby f.h-f.s. » Mon May 01, 2017 6:43 am

if your stack isn't big enough, your variable cant be allocated or will overflow your task stack. You have a overflow there, you declare a big two dimensional array in a function that overwrites that array. Right?

commarmi76
Posts: 7
Joined: Sun Apr 23, 2017 10:06 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby commarmi76 » Mon May 01, 2017 9:35 pm

f.h-f.s. wrote:if your stack isn't big enough, your variable cant be allocated or will overflow your task stack. You have a overflow there, you declare a big two dimensional array in a function that overwrites that array. Right?
Yes. Is there any method to check this kind of overflow implemented in freertos ?
I got crazy trying to debug this issue.

ESP_Sprite
Posts: 8921
Joined: Thu Nov 26, 2015 4:08 am

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby ESP_Sprite » Tue May 02, 2017 1:15 am

We have stack canaries, as well as the option to put a watchpoint on the end of the stack; see menuconfig for these things. Because of pretty fundamental reasons, this only can catch writes to the last 64 or so bytes of the stack, so if you allocate an array on the stack and don't write to all entries, you can still manage to corrupt memory. In the general case, this usually helps catching those issues however.

f.h-f.s.
Posts: 214
Joined: Thu Dec 08, 2016 2:53 pm

Re: pvPortMallocCaps with MALLOC_CAP_32BI and memset

Postby f.h-f.s. » Tue May 02, 2017 6:12 am

You can also use malloc and free instead of declaring a array.
overflow canaries are definitely a must have.

Who is online

Users browsing this forum: No registered users and 115 guests