IDF and ESPTOOL_PY Licence Clarification Request

Andrzej
Posts: 7
Joined: Tue Mar 10, 2020 8:03 pm

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby Andrzej » Wed Mar 18, 2020 5:56 pm

Thank you. Yes, I've just found a descriptin of the flash upload protocol: https://github.com/espressif/esptool/wi ... l-Protocol

After porting it (mentioned functions, but also those filesystem related) into STM32 project and an attempt to run it with ESP8266 board there are a couple of observations.serial_flash_example_main.c / esp_flash_new_image() is an entry point here. There are just slight defferences between ESP8266 and ESP32 within the protocol and it make sense (as one of next steps when working on the code) to add something like "chip profile" (or the loader_config.h file ?) into the C flash uploader code, so it can work for both chips.

1) some commands are not supported by ESP8266. Those found so far are: CHANGE_BAUDRATE, SPI_FLASH_MD5 . Commenting out the section of the example code works. That's documented on above www page, but taking the STM32 example and using it for ESP8266 (this is what I did) will just not work. Some fine-tinning is just needed.

2) Msg structure - slight differences between ESP8266 vs ESP32:
typedef struct __attribute__((packed))
{
command_common_t common;
uint32_t configuration;
//uint32_t zero; // ESP32 ROM only
} spi_attach_command_t;

typedef struct __attribute__((packed))
{
uint8_t failed;
uint8_t error;
// uint8_t reserved_0; // ESP32 ROM only
// uint8_t reserved_1; // ESP32 ROM only
} response_status_t;

3) Serial Speed = 74800 rather then 115200 ( ESP Crystal Freq = 26 MHz vs 40MHz ) -> good to know, as there may be there may be defferences between board types.

4) STM32 is faster then ESP32, so implementation (part of the code port) of loader_port_serial_read() needs some retry (when polling for a RX data), so it does not exit prematurely.

5) Flash erase command works in the exmple for 4kB file/block, but the function loader_flash_begin_cmd() fails when trying to load ~450kB file. Reason is: the flash erase operation takes just more time for the bigger block, while check_response() exits prematurely with fail status. More retry cycles (longer polling for a response from a chip) makes the function working.

Finally the code upload is reported as pass bo a response for a chip (WOW - some little success finally :-) ), but I have not succeeded to boot from the newly uploaded code yet. It;s possible tehre may be some other trick to do. Kindly please to share hints if any are known.

I appreciate sharing the solution. Making it from scrach (everyone separatelly) would be way harder, and here there is at least some structure almost working "off the shelf", so it shall be able to polish it a little, and it will work finally :-).

Thanks,
Andrzej

Andrzej
Posts: 7
Joined: Tue Mar 10, 2020 8:03 pm

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby Andrzej » Sat Mar 21, 2020 9:30 pm

Ok, one more thing was to comment out loader_spi_attach_cmd(), as it was causing some error on ESP side. It's possible the spi config value needs to be more sophisticated than the default proposed. It looks to be somehow working w/o the function.

esp_loader_error_t esp_loader_connect(esp_loader_connect_args_t *connect_args)
{
...
// return loader_spi_attach_cmd(SPI_PIN_CONFIG_DEFAULT);
return ESP_LOADER_SUCCESS;
}

The chip boots into bootloader and into user1 application code on ESP8266 correctly now. Also both init_data and blank.bin files are loaded.

I observe quite frequent disconnections from host wifi network. I think it was working properly before the upgrade. Maybe the SPI config needs some fine-tunning for performance boost... , hard to say yet what is a reason.

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby ESP_Angus » Sun Mar 22, 2020 11:39 pm

Andrzej wrote:
Sat Mar 21, 2020 9:30 pm
Ok, one more thing was to comment out loader_spi_attach_cmd(), as it was causing some error on ESP side. It's possible the spi config value needs to be more sophisticated than the default proposed. It looks to be somehow working w/o the function.

esp_loader_error_t esp_loader_connect(esp_loader_connect_args_t *connect_args)
{
...
// return loader_spi_attach_cmd(SPI_PIN_CONFIG_DEFAULT);
return ESP_LOADER_SUCCESS;
}

The chip boots into bootloader and into user1 application code on ESP8266 correctly now. Also both init_data and blank.bin files are loaded.
This SPI attach command is not present on the ESP8266, disabling it is correct.
Andrzej wrote:
Sat Mar 21, 2020 9:30 pm
I observe quite frequent disconnections from host wifi network. I think it was working properly before the upgrade. Maybe the SPI config needs some fine-tunning for performance boost... , hard to say yet what is a reason.
It is possible that the header written to flash needs to be changed in order to increase SPI performance, esptool.py may rewrite this header with new values based on the command line arguments given to "write_flash" command, but the serial loader library doesn't do this. You can still make sure the correct header is set when you generate the .bin file for the ESP8266 firmware, though, and then the serial loader library will copy it to the flash without modification.

If unsure then suggest reading out a copy of the flash written by esptool.py and a copy written using the loader library, and look for any differences in the binary.

Andrzej
Posts: 7
Joined: Tue Mar 10, 2020 8:03 pm

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby Andrzej » Mon Mar 23, 2020 6:49 pm

Thank you. In paralel I've also found HHD Device Monitoring Studio ( https://www.hhdsoftware.com/Downloads/d ... ing-studio ). Its free version is good enough to sniff / log / depict all Serial Port communication under windows and compare with what esptool.py does (with skip 'stub' download option = using ROM only) vs. what the C does , espacially in the fash download startup phase.

As of reading the flash: good idea in genaral. It would be soldering in and out esp8266 doughter board in my case or playing with voltage level / usb external converters (connecting the to the main board) in my case though. Also the C code does not have utilities for reading yet (as far as I see), so I will need to posptone it for now.

Could you please share a word ;) on what in particular and where is being written as of the SPI config is concerned ? The flashing phase looks to go through OK somehow. Is in possible binaries (generated from a template project from Arduino Studio 1.8.12) or stock ESP8266 AT Bin V1.6.2 binaries (or ROM?) can detect / overwrite / correct it after a startup, or the uploader needs to do it ?

ROM / bootloader says:

Code: Select all

  ets Jan  8 2013,rst cause:2, boot mode:(3,7)

load 0x40100000, len 2592, room 16 
tail 0
chksum 0xf3
load 0x3ffe8000, len 764, room 8 
tail 4
chksum 0x92
load 0x3ffe82fc, len 676, room 4 
tail 0
chksum 0x22
csum 0x22

2nd boot version : 1.7(5d6f877)
SPI Speed : 40MHz
SPI Mode : QIO
SPI Flash Size & Map: 4Mbit(256KB+256KB)
jump to run user1 @ 1000

rf cal sector: 123
freq trace enable 1
rf[112] : 03
rf[113] : 00
rf[114] : 01
w_flash

SDK ver: 2.2.1(6ab97e9) compiled @ Jun  7 2018 13:59:14
phy ver: 1136_0, pp ver: 10.2
I've actually begun to like playing w/ this ;) , espacially since a project using custom esp8266 binaries works way better (more stable, just performing better) if a web server's static content (files) are sitting in SPIFFS on esp8266 (skipping AT commands for transferring it from host MCU). :)

ESP_Angus
Posts: 2344
Joined: Sun May 08, 2016 4:11 am

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby ESP_Angus » Tue Mar 24, 2020 7:34 am

Andrzej wrote:
Mon Mar 23, 2020 6:49 pm
Could you please share a word ;) on what in particular and where is being written as of the SPI config is concerned ? The flashing phase looks to go through OK somehow. Is in possible binaries (generated from a template project from Arduino Studio 1.8.12) or stock ESP8266 AT Bin V1.6.2 binaries (or ROM?) can detect / overwrite / correct it after a startup, or the uploader needs to do it ?
There are two bytes at the beginning of the .bin image which encode the flash mode (QIO, DIO, etc), the flash speed (80MHz, 40MHz, etc) and the flash size (1MB, 2MB, etc):
https://github.com/espressif/esptool/wi ... age-Format

When the .bin file is generated then some settings will be written to these fields. However, esptool can update these fields when it writes the image to flash (using command line options like --flash-mode, --flash-size, etc).

You can see the current values in the boot log you posted:
Andrzej wrote:
Mon Mar 23, 2020 6:49 pm

Code: Select all

SPI Speed : 40MHz
SPI Mode : QIO
SPI Flash Size & Map: 4Mbit(256KB+256KB)
If this looks correct then you don't need to change anything at all.

However, if you wanted a 80MHz flash speed and you're only seeing 40MHz here then this may explain the slower performance. You can check whatever generated the .bin file to make sure it's being generated for 80MHz SPI flash speed.

For more details you may need to look to other resources, this forum is for ESP32 not ESP8266.

Andrzej
Posts: 7
Joined: Tue Mar 10, 2020 8:03 pm

Re: IDF and ESPTOOL_PY Licence Clarification Request

Postby Andrzej » Thu Mar 26, 2020 6:44 pm

Thanks a lot :idea: - it helps !

Who is online

Users browsing this forum: Baidu [Spider] and 190 guests