Hacking partitions relocation

[code4sex]

As known from IDF docs Espressif uses the following flash locations:

0x0000..0x1000 = stuff for secure boot
0x1000..0x6000+ = 2nd bootloader
0x6000+..0x7000+ = partition_table
0x7000+..and above = partitions

Note, 2nd-stage bootloader location is hardcoded in 1st bootloader ROM and always starts at 0x1000.
The size of the 2nd bootloader depends upon the log verbosity setting and, probably, IDF version.
Basically, with Bootloader log verbosity set to "No output" and Flash encryption enabled (dev mode) the size of the bootloader.bin file is 21K bytes something.
In my particular case (IDF 5.0) the size is 21440 (0x53C0) bytes.
Such a bootloader.bin file will require 0x6000 bytes in flash, hence the "0x6000+" record in the list above.
The "+" sign indicates that the size might increase with a more detailed log verbosity parameter set.

The partition table is always 96*32 = 3072 (0xC00) bytes and will fit in 4K sector.

Also, IDF docs say that the first sector in flash is unused if Secure boot option is not engaged (https://docs.espressif.com/projects/esp ... artup.html)

Hence it's a good idea to shift partition_table to 0x0000 if secure boot is not required.

In order to do so one needs to follow several steps:

STEP #1: SDKconfig -> make sure secure boot IS NOT activated

STEP #2: SDKconfig -> Partition table -> Offset of partition table = 0

STEP #3: Compile the project with ESP-IDF and get critical error saying stupid things about bootloader and partition_table overlapping

STEP #4: Open "ESP-IDF\components\partition_table\check_sizes.py" file, scroll down to main() and find and comment the hell out the following code:

Code: Select all

#    if args.check_target == 'bootloader':
#        check_bootloader(args.offset, int(args.bootloader_offset, 0), args.bootloader_binary)
#    else:
#        check_partition(args.type, args.subtype, args.partition_table, args.binary)

Instead, write the following one:

Code: Select all

    if args.check_target == 'partition':
         check_partition(args.type, args.subtype, args.partition_table, args.binary)

This will make the script ignore the bootloader check and will not blow your mind saying that the space available for bootloader is negative. The author of the original script assumed the partition_table is strictly located AFTER bootloader and no special checks are performed prior to offset calculation.

STEP #5: From now on you won't get any bootloader errors and can recompile the project. However, you should always keep in mind that no bootloader checks are performed so it is you who must ensure no overlapping with other partitions occur.


Further "hacking":

Instead of partition_table you can place any arbitrary 4K partition @ 0x0000.
To do so open "ESP-IDF\components\partition_table\gen_esp32part.py" and find the following code (refer to my ### comments inside):

Code: Select all

# fix up missing offsets & negative sizes
        last_end = offset_part_table + PARTITION_TABLE_SIZE  # first offset after partition table
        for e in res:
            if e.offset is not None and e.offset < last_end:
                ### Getting here means the current partition starts BEFORE the end of partition table
                ### For some reason Espressif takes this for error...
                ### However, a partition in question can easily START and END before the partition table especially when IDF allows for table relocation... This is a bug!
                
                if e == res[0]:
                    raise InputError('CSV Error: First partition offset 0x%x overlaps end of partition table 0x%x'
                                     % (e.offset, last_end))
                else:
                    raise InputError('CSV Error: Partitions overlap. Partition at line %d sets offset 0x%x. Previous partition ends 0x%x'
                                     % (e.line_no, e.offset, last_end))
            if e.offset is None:
                pad_to = get_alignment_for_type(e.type)
                if last_end % pad_to != 0:
                    last_end += pad_to - (last_end % pad_to)
                e.offset = last_end
            if e.size < 0:
                e.size = -e.size - e.offset
            last_end = e.offset + e.size

This is the proper one (copy/paste-able):

Code: Select all

	# fix up missing offsets & negative sizes
        last_end = offset_part_table + PARTITION_TABLE_SIZE  # first offset after partition table
        for e in res:
            if e.offset is not None and e.offset >= offset_part_table and e.offset < last_end:
                if e == res[0]:
                    print(e)
                    raise InputError('CSV Error: First partition (offset 0x%x) overlaps the partition table' % (e.offset))
                else:
                    raise InputError('CSV Error: Partitions overlap. Partition at line %d sets offset 0x%x. Previous partition ends 0x%x'
                                     % (e.line_no, e.offset, last_end))
            if e.offset is None:
                pad_to = get_alignment_for_type(e.type)
                if last_end % pad_to != 0:
                    last_end += pad_to - (last_end % pad_to)
                e.offset = last_end
            if e.size < 0:
                e.size = -e.size - e.offset
            last_end = e.offset + e.size

After editing you won't get those idiotic error messages from a script that got scared of a partition starting prior to partition_table end!

PS. python sucks. bullshit language. my opinion.