Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

article: How to migrate from IDF Bootloader to MCUboot using ESP Self-Reflasher #363

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

almir-okato
Copy link
Contributor

This article shows how to use the ESP Self-Reflasher component can be used to migrate an ESP32 embedded with an
IDF-based application (OTA capable) + IDF bootloader to MCUboot compatible application + MCUboot bootloader.

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 865181c to 856d026 Compare December 16, 2024 15:12
Copy link
Contributor

@fdcavalcanti fdcavalcanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For me the HOST_IP thing seems fine. But if I'm not familiar with networking, I would wonder if there is any specific port that I should use.

Can you put an example like 192.168.0.10:3000? Some generic IP and port.

All in all, looks great to me, nice work!

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 856d026 to 8cb59dc Compare December 17, 2024 14:20
@almir-okato
Copy link
Contributor Author

For me the HOST_IP thing seems fine. But if I'm not familiar with networking, I would wonder if there is any specific port that I should use.

Can you put an example like 192.168.0.10:3000? Some generic IP and port.

All in all, looks great to me, nice work!

Got your point, I added some examples for that now. Thanks!

@almir-okato
Copy link
Contributor Author

@pedrominatel could you please take a look?

Copy link
Contributor

@fdcavalcanti fdcavalcanti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 8cb59dc to 26facdb Compare December 17, 2024 20:48
@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch 2 times, most recently from a51b04a to f86d708 Compare December 18, 2024 20:58
@almir-okato
Copy link
Contributor Author

Hi @f-hollow, could you please take a look as well?
It seems I cannot add reviewers directly on the PR neither add labels like "needs review".
Thanks in advance!

@horw horw requested review from f-hollow and removed request for sylvioalves December 20, 2024 12:01
@f-hollow
Copy link
Collaborator

f-hollow commented Dec 23, 2024

Hi @f-hollow, could you please take a look as well? It seems I cannot add reviewers directly on the PR neither add labels like "needs review". Thanks in advance!

Hi Almir! Sorry for my slow reply. I will review your PR today or tomorrow.

Yeah, you cannot add reviewers or labels now. We will think about how to make it better and more convenient for everybody.

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from f86d708 to 66ffdb9 Compare December 23, 2024 16:12
Copy link
Collaborator

@f-hollow f-hollow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@almir-okato Thank you for writing this article!

Sorry for my numerous comments, but addressing the mentioned issues will make the article more attractive and easier to follow.

idf.py build
```

### Building and flashing the IDF application
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same here, looks like this section has a procedure or a how-to, please format it as a procedure with distinct numbered steps and indented code blocks to show that they belong to this particular step.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we standardize the ESP-IDF instead of IDF? I see many references to it just as IDF and not as ESP-IDF.

@f-hollow
Copy link
Collaborator

Regarding heading titles, suggest using more consistent capitalization.

You mostly use the sentence-style capitalization, which is also most common these days, such as "Building and flashing the IDF application". However, you also use the heading-style capitalization at times, such as "Step-by-step Guide". Suggest replacing it with "Step-by-step guide".

Also, I am not sure if "Target Reflashing Image" or "Reflashing application" should be capitalized like these, unless these are proper nouns or terms with established capitalization.

So please consider making heading capitalization more consistent.

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 66ffdb9 to 59877ea Compare January 14, 2025 04:50
@almir-okato almir-okato requested a review from f-hollow January 14, 2025 04:52
@almir-okato
Copy link
Contributor Author

Hi @f-hollow, sorry for the delay.
I've tried to apply your suggestions, could you please take a look again?

Copy link
Member

@pedrominatel pedrominatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please rebase to main? This will fix the link check error in the CI.

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 59877ea to 7e61b4a Compare January 14, 2025 16:11
Copy link
Member

@pedrominatel pedrominatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @almir-okato for the updated version. Please find my comments.


6. Only for illustration purposes, this guide is using simple HTTP requests, so enable the `Allow HTTP for OTA` option in the `Component config -> ESP HTTPS OTA`.

{{< alert icon="eye" >}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid using Alert shortcode when you have a code block inside.

# e.g. sudo python -m http.server -b 192.168.0.100 8070
```

{{< alert icon="eye" >}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue here. The render does not look good.


#### `boot_swap_download_example`

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add code block as text. This will reduce the font size for this block, and the copy button will be added.


#### `boot_swap_embedded_example`

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add code block as text.

{{< alert icon="eye" >}}
If the `boot_swap_embedded_example` was used, define a custom partition table in `Partition table` menu. Create the `partitions.csv` file with the following content:

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add code block as text.


4. In the `Example Configuration` menu, set the `firmware upgrade url endpoint` with the host IP, port and the Reflasher application binary name (e.g. `http://192.168.0.100:8070/boot_swap_download.bin`):

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add code block as text.


4. Enter menu `Example Configuration -> Bootloader reflash configuration` and set the `Bootloader bin url endpoint` with the host IP, port and the bootloader binary name (e.g. `http://192.168.0.100:8070/mcuboot-esp32.bin`):

```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do the same for all blocks without the type.


Besides initializing the system, the bootloader may be responsible for handling safe and secure firmware updates. However, it typically cannot update itself, either by design (usually due to security and safety reasons) or due to device restrictions, such as flash bootloader region protection. Also in most of the scenarios, the device is not physically accessible, thus a direct serial flashing is not practical.

Support for Espressif chips on platforms other than **ESP-IDF**, like **NuttX RTOS** and **Zephyr RTOS** keeps improving and increasing, which means more interesting options for either having a flexible compound when projecting, or if facing the need of a firmware migration. However there is no standard support for ESP-IDF bootloader on the mentioned RTOS, so in the case of an eventual migration from **ESP-IDF** to one of them, the bootloader would need to be replaced with **MCUboot** bootloader, for example, that is one of the options available for booting either **NuttX RTOS** or **Zephyr RTOS** on Espressif chips.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider adding the links to the highlighted platforms.

@@ -0,0 +1,834 @@
---
title: "How to migrate from IDF Bootloader to MCUboot using ESP Self-Reflasher"
date: 2024-12-16
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change the date to today and move this article from 2024/12 to the 2025/01 folder.

@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from 7e61b4a to cdb48d0 Compare January 15, 2025 21:10
Copy link
Collaborator

@f-hollow f-hollow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@almir-okato Thank you for implementing my comments. Please check a few more suggestions for the updated parts. Otherwise, LGTM!

- Better support from vendor or community;
- Security;

Also it is not uncommon to migrate a platform out of necessity driven by constraints or issues faced times after implementation, thus choosing a flexible platform is often a good consideration when projecting a solution.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Also it is not uncommon to migrate a platform out of necessity driven by constraints or issues faced times after implementation, thus choosing a flexible platform is often a good consideration when projecting a solution.
Also it is not uncommon to migrate to another platform out of necessity driven by constraints or issues that appear after implementation, thus choosing a flexible platform is often a good consideration when designing a solution.


Also it is not uncommon to migrate a platform out of necessity driven by constraints or issues faced times after implementation, thus choosing a flexible platform is often a good consideration when projecting a solution.

In Embedded Systems, migration is a quite important subject that may emerge as result of an ongoing production decision, or serve as a factor contributing to design choices at the project stage. Hardware-wise or firmware-wise, any change of that kind is risky, so it requires good planning, controlled testing, and careful execution.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In Embedded Systems, migration is a quite important subject that may emerge as result of an ongoing production decision, or serve as a factor contributing to design choices at the project stage. Hardware-wise or firmware-wise, any change of that kind is risky, so it requires good planning, controlled testing, and careful execution.
In Embedded Systems, migration is quite an important subject that may emerge as a result of an ongoing production decision, or serve as a factor contributing to design choices at the project stage. Hardware-wise or firmware-wise, any change of that kind is risky, so it requires good planning, controlled testing, and careful execution.


Besides initializing the system, the bootloader may be responsible for handling safe and secure firmware updates. However, it typically cannot update itself, either by design (usually due to security and safety reasons) or due to device restrictions, such as flash bootloader region protection. Also in most of the scenarios, the device is not physically accessible, thus a direct serial flashing is not practical.

Support for Espressif chips on platforms other than [**ESP-IDF**](https://idf.espressif.com/), like [**NuttX RTOS**](https://nuttx.apache.org/) and [**Zephyr RTOS**](https://zephyrproject.org/) keeps improving and increasing, which means more interesting options for either having a flexible compound when projecting, or if facing the need of a firmware migration. However there is no standard support for **ESP-IDF** bootloader on the mentioned RTOS, so in the case of an eventual migration from **ESP-IDF** to one of them, the bootloader would need to be replaced with [**MCUboot**](https://docs.mcuboot.com/) bootloader, for example, that is one of the options available for booting either **NuttX RTOS** or **Zephyr RTOS** on Espressif chips.
Copy link
Collaborator

@f-hollow f-hollow Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph looks a bit too long. If you look closer, it has two ideas: (1) Interesting options that NuttX and Zephyr bring, (2) Issues with ESP-IDF bootloader and migration to MCUboot. Suggest splitting this into two paragraphs as proposed below.

Also, improving support for NuttX or Zephyr does not automatically imply that you have more interesting options (keeps improving and increasing, which means more interesting options ...). Suggest slightly changing the narrative to make it more logical.

Another thing, it is not clear what having a flexible compound when projecting means. Does it provide a more flexible platform or framework that facilitates designs with more alternatives? Some updates are proposed below, please see if it expresses your original thought.

Suggested change
Support for Espressif chips on platforms other than [**ESP-IDF**](https://idf.espressif.com/), like [**NuttX RTOS**](https://nuttx.apache.org/) and [**Zephyr RTOS**](https://zephyrproject.org/) keeps improving and increasing, which means more interesting options for either having a flexible compound when projecting, or if facing the need of a firmware migration. However there is no standard support for **ESP-IDF** bootloader on the mentioned RTOS, so in the case of an eventual migration from **ESP-IDF** to one of them, the bootloader would need to be replaced with [**MCUboot**](https://docs.mcuboot.com/) bootloader, for example, that is one of the options available for booting either **NuttX RTOS** or **Zephyr RTOS** on Espressif chips.
Support for Espressif chips on platforms other than [**ESP-IDF**](https://idf.espressif.com/), like [**NuttX RTOS**](https://nuttx.apache.org/) and [**Zephyr RTOS**](https://zephyrproject.org/) keeps improving. It brings more interesting options to the table, such as offering a more flexible platform that provides more design choices or offering more flexibility in firmware migration.
However there is no standard support for **ESP-IDF** bootloader on the mentioned RTOS, so in the case of an eventual migration from **ESP-IDF** to one of them, the bootloader would need to be replaced with [**MCUboot**](https://docs.mcuboot.com/) bootloader, for example, that is one of the options available for booting either **NuttX RTOS** or **Zephyr RTOS** on Espressif chips.


Support for Espressif chips on platforms other than [**ESP-IDF**](https://idf.espressif.com/), like [**NuttX RTOS**](https://nuttx.apache.org/) and [**Zephyr RTOS**](https://zephyrproject.org/) keeps improving and increasing, which means more interesting options for either having a flexible compound when projecting, or if facing the need of a firmware migration. However there is no standard support for **ESP-IDF** bootloader on the mentioned RTOS, so in the case of an eventual migration from **ESP-IDF** to one of them, the bootloader would need to be replaced with [**MCUboot**](https://docs.mcuboot.com/) bootloader, for example, that is one of the options available for booting either **NuttX RTOS** or **Zephyr RTOS** on Espressif chips.

**MCUboot** bootloader is an open source secure bootloader that handles fault tolerant updates and image verification for authenticity and integrity. **MCUboot** project was designed aiming to have a good coverage of bootloader capabilities so it could be attractive to be ported and used within more platforms.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**MCUboot** bootloader is an open source secure bootloader that handles fault tolerant updates and image verification for authenticity and integrity. **MCUboot** project was designed aiming to have a good coverage of bootloader capabilities so it could be attractive to be ported and used within more platforms.
**MCUboot** bootloader is an open source secure bootloader that handles fault-tolerant updates and image verification for authenticity and integrity. **MCUboot** was designed to provide good coverage of bootloader capabilities so it could be attractive to be ported to and used with more platforms.


However, as said before, migration is driven by variable reasons according to each project, thus it is not the goal of this article to discuss or compare each platform.

This tutorial aims to demonstrate how **ESP Self-Reflasher** component can be used to migrate an ESP-IDF-bootloader-based application (OTA capable) to a MCUboot-based-bootloader application in case of an eventual need. The **ESP Self-Reflasher** component is used on an middle step application for enabling the device to "reflash" its own firmware.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This tutorial aims to demonstrate how **ESP Self-Reflasher** component can be used to migrate an ESP-IDF-bootloader-based application (OTA capable) to a MCUboot-based-bootloader application in case of an eventual need. The **ESP Self-Reflasher** component is used on an middle step application for enabling the device to "reflash" its own firmware.
This tutorial aims to demonstrate how **ESP Self-Reflasher** component can be used to migrate an ESP-IDF-bootloader-based application (OTA capable) to an MCUboot-based-bootloader application in case of an eventual need. The **ESP Self-Reflasher** component is used as a middle step application for enabling the device to "reflash" its own firmware.

idf.py build
```

#### Example 2: Embedd the target reflashing image
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Example 2: Embedd the target reflashing image
#### Example 2: Embed the target reflashing image

Comment on lines 82 to 95
1. With the **NuttX** workspace ready, prepare the NSH (NuttShell) configuration with **MCUboot** compatibility, which is available under the `mcuboot_nsh` defconfig (premade application configuration for a board, more information [here](https://nuttx.apache.org/docs/latest/quickstart/configuring.html)). This is the build that will be used as the *final target* of the reflashing process.

```bash
cd <NUTTX_DIR>
./tools/configure.sh esp32-devkitc:mcuboot_nsh
```

You can also manually configure other **NuttX** applications to be built with **MCUboot** compatibility:

- In your **NuttX** workspace, enter the menuconfig:
```bash
make menuconfig
```
- In the menu `System Type -> Bootloader and Image Configuration`, enable the option `Enable MCUboot-bootable format`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last step to make formatting intuitive is to fix the indentation.

Likewise, please fix the indentation in all other procedure steps in the article.

Suggested change
1. With the **NuttX** workspace ready, prepare the NSH (NuttShell) configuration with **MCUboot** compatibility, which is available under the `mcuboot_nsh` defconfig (premade application configuration for a board, more information [here](https://nuttx.apache.org/docs/latest/quickstart/configuring.html)). This is the build that will be used as the *final target* of the reflashing process.
```bash
cd <NUTTX_DIR>
./tools/configure.sh esp32-devkitc:mcuboot_nsh
```
You can also manually configure other **NuttX** applications to be built with **MCUboot** compatibility:
- In your **NuttX** workspace, enter the menuconfig:
```bash
make menuconfig
```
- In the menu `System Type -> Bootloader and Image Configuration`, enable the option `Enable MCUboot-bootable format`
1. With the **NuttX** workspace ready, prepare the NSH (NuttShell) configuration with **MCUboot** compatibility, which is available under the `mcuboot_nsh` defconfig (premade application configuration for a board, more information [here](https://nuttx.apache.org/docs/latest/quickstart/configuring.html)). This is the build that will be used as the *final target* of the reflashing process.
```bash
cd <NUTTX_DIR>
./tools/configure.sh esp32-devkitc:mcuboot_nsh
```
You can also manually configure other **NuttX** applications to be built with **MCUboot** compatibility:
- In your **NuttX** workspace, enter the menuconfig:
```bash
make menuconfig
```
- In the menu `System Type -> Bootloader and Image Configuration`, enable the option `Enable MCUboot-bootable format`

#### `boot_swap_download_example`

```text
I (29) boot: ESP-IDF v5.1.4 2nd stage bootloader
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The execution outputs are quite long. Are all the lines important?

It is unlikely that users (developers) will be comparing each line of the given output with their output. Isn't it more convenient to provide only important parts and say what to pay attention to or what can go wrong?

Something like this as an example?

I (37) boot.esp32: SPI Speed      : 40MHz
I (42) boot.esp32: SPI Mode       : DIO
I (47) boot.esp32: SPI Flash Size : 4MB
...

If you still want to keep longer code blocks, you can use the details section like this:

Rendered version

Click me
I (37) boot.esp32: SPI Speed      : 40MHz
I (42) boot.esp32: SPI Mode       : DIO
I (47) boot.esp32: SPI Flash Size : 4MB
...

Markdown syntax:

<details>
  <summary>Click me</summary>

```text
I (37) boot.esp32: SPI Speed      : 40MHz
I (42) boot.esp32: SPI Mode       : DIO
I (47) boot.esp32: SPI Flash Size : 4MB
...
```
</details>

…-Reflasher

This article shows how to use the ESP Self-Reflasher component
can be used to migrate an ESP32 embedded with an
IDF-based application (OTA capable) + IDF bootloader to
MCUboot compatible application + MCUboot bootloader

Signed-off-by: Almir Okato <[email protected]>
@almir-okato almir-okato force-pushed the esp_self_reflasher_how_to branch from cdb48d0 to 6a3d74f Compare January 16, 2025 18:00
@almir-okato
Copy link
Contributor Author

@f-hollow and @pedrominatel thanks for all your valuable inputs!
I've updated the PR, hope it's ok!

Copy link
Member

@pedrominatel pedrominatel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thank you very much @almir-okato and @f-hollow for the review.

@pedrominatel pedrominatel merged commit 201c097 into espressif:main Jan 16, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants