.png){kind=small}
Dasharo firmware starts to support capsule updates. This post looks into what it is, why it is useful, and what has led to this.
Capsule updates for MSI boards
The first boards to get the support are MSI PRO Z690-A (v1.1.4) and MSI PRO Z790-P (v0.9.2) (all their WIFI/DDR4/DDR5 variants). Do note that the functionality cannot be used to update from previous firmware releases for these boards. Only future updates can leverage this update method unless you use it to “reinstall” newest releases as an experiment.
Worth mentioning right away that this work was funded by NLnet Foundation in the interest of making updates of open source firmware a more pleasant experience. More details about the project can be found here.
Firmware updates
Firmware is a crucial software component without which a device can’t function properly, in fact, most likely can’t function at all. Like anything, firmware is subject to defects and, due to its integral role for a device, a firmware defect can have a significant impact on its operation. A bad firmware can be subject to an attack or sometimes cause a physical damage to a device. Even in the absence of any issues, an update might still be desirable to provide more configuration options or support wider range of hardware.
For these and other reasons firmware updates are a necessity. As devices get more advanced, firmware defects become more likely, thus further increasing the importance of keeping firmware up-to-date.
Unfortunately, carrying out a firmware update is a relatively difficult task that itself can go wrong and lead to a broken device. This is hard to address in full, but some update methods are more user-friendly than others and update capsules are an example of moving in the direction of greater convenience to end users.
Firmware update methods
Let’s have a quick overview of update methods to have the necessary context for discussing capsule updates.
First of all, an update can be performed either internally or externally. An external update often requires a special device (see this kit for an idea) capable of updating a flash chip which contains the firmware. The device gets connected to a host machine (so you need a second PC/laptop) from which the chip is updated via a specialized software (e.g., flashrom or flashprog).
An internal update means that the system applies an update to itself, which can happen in at least two different ways:
- By the firmware (e.g., after entering its setup interface and picking a file containing an update).
- By an application running in an operating system (a generic one like
flashromor a one provided by system’s manufacturer).
What are UEFI capsules
UEFI Update Capsules (also update capsules, capsule updates or just capsules) are a way of delivering firmware updates to UEFI firmware. A file format in which such updates are provided is known by the same name.
Capsule updates belong to an internal kind of updates performed by firmware itself. The details are defined in UEFI specification, but no need to study it to be able to understand how capsules work. In a nutshell:
- Firmware actively participates in the update process:
- An update can be validated and rejected if it can’t work on a specific hardware, thus preventing bricking the device.
- Current settings and other kinds of user data can be preserved.
- Regular security measures that cause other kinds of internal updates to fail can be lifted automatically.
- Capsule updates can be performed without an OS.
- There are several ways for delivering updates:
- Via
CapsuleApp.efiin UEFI Shell (or really using any EFI application). - By an OS if it supports this kind of communication with the firmware.
- Via
- There are several ways in which a capsule can be delivered to a firmware:
- In-RAM. A capsule is left in RAM while the system reboots retaining RAM’s contents. This is what got implemented.
- On-disk. A firmware is tasked with reading a capsule from a designated directory on ESP (EFI System Partition). This can be added in the future.
- There are several pre-defined kinds of capsules:
- UX (User eXperience) capsules, responsible for providing some on-screen information about an ongoing update.
- FMP (Firmware Management Protocol) capsules, these are the ones which carry the updated firmware.
The advantages of capsules are covered more verbosely in the documentation. One extra thing worth mentioning is that capsules are already used by a number of firmware vendors (even though it might not be obvious). This convergence on a format allows for a more unified distribution and handling of updates, in particular through LVFS/fwupd.
LVFS is not restricted to capsule updates alone and works with plain flash
images just fine. However, use of capsules comes with their advantages such as
data migration. Doing it with plain images would require modifying fwupd,
while capsules provide the same functionality regardless of how they are
applied.
As for disadvantages, because an update is carried out without user interactions and before any OS is loaded an unexpected error can result in an unbootable system. That is always a possibility, yet with some other internal flashing methods it might be possible to keep the machine running while trying to restore flash contents to a reasonable state.
How a capsule update works
In general, it is possible for a capsule to be processed immediately or after a system reboot. System firmware updates fall into the second category, so a successful update looks as follows:
- Pass capsule to the firmware (see below for several ways to do it).
- Reboot (can happen automatically as part of the first step).
- Firmware detects presence of a capsule, runs all the necessary checks and applies the capsule while the user looks at a progress bar. Like with any firmware update, it’s very important to not power off the machine during this process.
- The machine reboots again but this time boots into a different firmware.
An unsuccessful update looks very similar, but it’s still an old version at the last step (assuming nothing went horribly wrong and the system can still start).
How to perform a capsule update
One way of doing it is running CapsuleApp.efi in a UEFI shell. This is
thoroughly explained in the documentation.
DTS should be able to use capsules as well, but there the process will mostly be hidden from the user.
A Linux-specific way is to write a capsule into a special device which is typically not present by default:
|
|
The next section of the post will be a bit more technical. Skip everything until What’s next if you’re not interested in this part.
Changes to firmware components (coreboot and EDK2) and difficulties
Publication of ESRT
ESRT (EFI System Resource Table) is a way in which a UEFI firmware declares versions of firmware components. UEFI capsules target a specific entry in the table and can only be applied if a suitable entry is reported by the currently running firmware.
The ESRT is exposed at /sys/firmware/efi/esrt/entries/ in Linux and while you
can view some information there it’s mostly machine-oriented.
Collecting pieces of capsules on boot
UEFI specification requires a capsule to be in a continuous virtual memory, but not in physical memory. This means that after a reboot capsules might need to be stitched together from a bunch of small pieces. The process is tedious, error prone and potentially insecure. In addition to that, a reconstructed capsule better not be placed in an area which might be needed in later boot phases or things will fall apart in ways which are hard to diagnose.
Memory accesses beyond 32-bit address space
When a capsule survives a reset, it can end up almost anywhere in memory. coreboot on x86 systems typically runs as a 32-bit program (64-bit mode exists but is experimental) which means that memory accesses beyond first 4 GiB of RAM require special handling. This was made possible by updating PAE-enabled page tables that describe a virtual address space below 4 GiB boundary to point at a location above it, so that a 32-bit code could work with arbitrary locations.
Accessing full flash from EDK2
When coreboot and EDK2 are used together, it’s coreboot’s job to deal with a flash chip. coreboot mediates flash accesses so that EDK2 could store its settings between boots in a dedicated part of the chip. When EDK2 performs an update process, it needs to be able to access the whole chip. This extended access when an in-RAM capsule is present had to be implemented.
A related subject is flash protection. Quite often at least part of the flash is marked read-only by the firmware to make it more resilient against attacks (or just buggy devices or software) that could try to modify it. Dasharo is no exception in this regard. The way capsule updates are dealing with it is by automatically activating Firmware Update Mode which lifts the protections for the duration of the next boot.
Early testing and debugging
When writing a parsing code, you want see how it behaves on artificial data of different kinds to develop some confidence in its operation. Testing firmware is not an easy task in general and use of a virtual machine (VM) can be of great help. This situation was no exception with the QEMU serving as a VM.
Dasharo already had an OVFM firmware image for some time before starting this
project. However, that firmware did not have coreboot in it and was using a
separate EDK2 package called OvmfPkg which is not employed by Dasharo
releases targeting hardware platforms. This has led to an effort of adding a
Dasharo variant targeting Q35. It shares code with hardware variants
where possible, allowing it to be used for tests or just to check out Dasharo
firmware in a VM.
Availability of Dasharo for Q35 made it possible to abuse QEMU’s “loader” device to place predefined values in memory and observe how they are being processed by the code. It remained useful later when development focus has shifted towards EDK2 and it became possible to test capsules in a VM.
Buggy capsule tooling
Working with capsules requires scripts that aid in assembling, disassembling and
analyzing of capsule files. EDK2 provides such a tool as part of its
BaseTools. However, the number of discovered bugs suggests that it’s
underutilized. Most of them are fixed by now thanks to this project. Many
more details on the use of GenerateCapsule.py are provided in the
documentation.
Hardware testing
Since MSI boards were the first to get the feature, that’s where it was validated on hardware. Manual tests are done as described by the aforementioned update guide, but that’s not the real tests. Real tests are run automatically via Robot Framework and follow a similar sequence but also take into account various corner cases, verify whether data was actually migrated or that the update screen looks as expected. The OSFV tests of the feature have their own documentation.
Intel ME vs. capsules
Intel Management Engine is stored on the same flash chip as the main system firmware, leading to the both components being often updated simultaneously. Because ME operates independently and periodically writes to flash, care needs to be taken to temporarily pause its operation to perform an update. The problem is that pausing ME takes a cold reboot which, unlike a warm reboot, resets contents of RAM thus leaving no chance for in-RAM capsules. Because firmware doesn’t know when a user will initiate an update, it can’t disable ME beforehand. This leads to a requirement for ME to be HAP-disabled by the user and enabled back when the update is done. This will, hopefully, be improved in the future.
ME can also be soft-disabled. This way of disabling it is not suitable for performing an update due to an extra ME state transition in coreboot which also requires a cold reboot.
Data migration
A set of settings available in a given firmware is likely to remain the same or grow in a more recent version. It’s nice to not have to recreate the same configuration after every update. The same goes about custom boot logo and other user-configurable parts of a firmware.
In practice this means copying data from SMMSTORE, COREBOOT and
BOOTSPLASH regions into updated firmware so it’s already suitably configured
on the first boot.
Update screen
Default implementation in EDK2 is very peculiar:
- Location of the progress bar is tied to the dimensions of boot logo.
- When determining dimensions, some pixels of the logo are completely ignored.
Boot logo can be customized by a user, leading to unpredictable look of the progress screen. This is why a simple replacement driver was made to display basic information and force predictable placement of the progress bar.
Graphical progress bar
As was mentioned above, progress bar depends on certain characteristics of a logo. In addition to that it’s very eager to abort. If it can’t fetch a logo image, the whole firmware update progress is aborted despite such a non-critical issue.
Some measures were taken to recover from a bad user-supplied logo (e.g., if it uses BMP of unsupported version), although thanks to customized update screen such errors shouldn’t be a problem during an update.
What’s next
Upstreaming
After the development is done changes made to open source projects are offered for inclusion to their respective maintainers. This is not exactly a future step as various changes were sent upstream several months ago and even got merged. Upstreaming is rarely a fast process so it doesn’t hurt to start it as soon as you have a working implementation. It also doesn’t always go as planned with some changes never making it upstream for various reasons, but you still try.
coreboot
- Announcement on the mailing list
- Capsule-specific code changes (in progress)
- Q35 changes (merged)
- 4 GiB cleanup (merged)
EDK2
- Announcement on the mailing list
- Main pull request (submitted)
- Pull request with tooling fixes (merged)
Capsule updates for other Dasharo variants
NovaCustom laptops are the next to support UEFI capsule updates! Stay tuned.
Summary
Project’s abstract states that it “aims to simplify the update process and enhance the user experience, providing a more reliable approach compared to complex flashrom-based updates”. While there are still things to improve, adding update capsules to modern MSI boards with coreboot firmware seems to have made strands in reaching the stated goal even though you might have to wait until the next firmware update to see it in action.
If you are looking to extend firmware with the features you need, similar to
how it was done in this case, schedule a call or drop an email at
contact<at>3mdeb<dot>com to discuss how 3mdeb can assist with that.
