Embedded Systems · Firmware Updates
Trial Boot, Confirmation, and Automatic Rollback
How a new image is allowed to run under supervision, what counts as proof that it works, and why the confirmation step has to come from the application rather than the bootloader.
A validated image is a complete, unmodified image. It is not a working one. Every check the bootloader can perform runs before the new code executes a single instruction, so the class of failures that matter most, such as an image that boots and then hangs, resets, or drives an output into a damaging state, is invisible to all of them. Trial boot is the mechanism that lets the device discover those failures and recover without a technician.
The gap between verification and correctness
Verification is a statement about bytes. Correctness is a statement about behaviour, and the only way to observe behaviour is to run the code. A bootloader cannot run the application and also remain able to recover from it, because once control transfers, the bootloader is no longer executing.
What the bootloader can do is arrange for the trial to be observable after the fact. It records, in durable metadata, that the image now running is on trial and how many attempts it has been given. It then transfers control. If the application reaches a point where it can assert that it is functioning, it writes a confirmation. If it never does, the next reset observes the trial record and decrements the attempt count, and when the count reaches zero the bootloader reverts to the previous image.
The design question is what the application’s assertion should consist of, because the mechanism is only as good as that signal.
The watchdog is not a confirmation
The tempting shortcut is to treat survival as proof. If the device is still running after thirty seconds, the image works. A timer in the bootloader cannot observe this directly, but a hardware watchdog can: the application services it, and an image that hangs fails to service it and resets. After enough resets, the trial count expires.
This catches hangs and crashes and misses everything else. An image that boots, services its watchdog from an idle loop, and never brings up its radio passes. An image whose main task is broken but whose watchdog task still runs passes. An image that drives a motor output to a fixed value and sits there passes, and the failure is mechanical rather than computational. The watchdog confirms liveness, which is the weakest useful property, and it is already doing that job independently of the update mechanism.
Worse, the watchdog confirmation can be actively wrong. An image whose watchdog is serviced by a timer interrupt will keep the device alive while the main loop is stuck, which converts a detectable failure into a silent one. The failure surfaces later as a device that responds to nothing.
Confirmation as an application obligation
The stronger design makes confirmation an explicit act by the application, performed only after the application has reached the state it considers operational. That state is domain-specific and cannot be inferred: a sensor node might confirm after its first successful measurement and uplink, a controller after its control loop has completed a full cycle with valid inputs.
The obligation has a cost that is easy to underestimate. The application must be able to write the confirmation, which means it needs access to the metadata region and a driver for it, and it must be able to do so before its own storage layer is necessarily initialised. The confirmation write is therefore a low-level operation duplicated in the application, and it must agree with the bootloader’s record format down to the byte.
A middle design records confirmation lazily. The application sets a flag in RAM, and a periodic task flushes it to metadata alongside other state. This avoids the early-boot dependency and introduces a window in which the device has confirmed but has not recorded it. A power loss in that window costs another trial boot, which is harmless if the trial count is greater than one. The design is sound as long as the trial count is not one.
/* Bootloader side: decide what to run, given a trial in progress.
Slot numbers are stored plus one, so 0 means "none" and 1 or 2 name
a slot; the application sets confirmed_slot and never touches trial. */
struct boot_state {
uint8_t trial_slot; /* slot under trial, 0 when no trial */
uint8_t trial_remaining; /* attempts left before reverting */
uint8_t confirmed_slot; /* set by the application, never by the bootloader */
};
static int select_slot(struct boot_state *s, uint8_t *out_slot)
{
if (s->trial_slot == 0) { /* no trial in progress */
*out_slot = s->confirmed_slot - 1;
return s->confirmed_slot != 0;
}
if (s->trial_remaining == 0) { /* trial exhausted: revert */
s->trial_slot = 0;
meta_append(s); /* record the decision before acting */
*out_slot = s->confirmed_slot - 1;
return s->confirmed_slot != 0;
}
s->trial_remaining--; /* consume one attempt, durably */
meta_append(s); /* before the image is entered */
*out_slot = s->trial_slot - 1;
return 1;
}The ordering in that function is the whole safety argument. The attempt is consumed and recorded before control transfers, so a reset at any point after the transfer finds the decremented count. A bootloader that decremented after the application returned would never decrement, because the application does not return. And a bootloader that decremented in RAM and wrote later would lose the decrement on exactly the resets it is trying to count.
Why the revert must be possible without a copy
Reverting is trivial when both slots hold complete images and the metadata names which one to boot: the bootloader changes the active slot and resets. The old image is still in its slot, untouched, and it becomes the running image again. This is the direct-XIP arrangement described in dual-bank slots and boot selection, and its value is precisely that rollback costs nothing but a metadata write.
In a swap arrangement the situation is harder, because the update process exchanged the contents of the two slots. After the swap, the old image is in the slot that used to hold the new one, so reverting means swapping back, which is a whole-image copy performed under the same power-loss constraints as the original swap. Implementations that swap at update time usually keep the pre-swap image recoverable by not erasing it until the new image is confirmed, which requires a third region or a swap log that can be replayed in either direction.
The revert also has to be distinguishable from a normal boot for the purposes of reporting. A device that rolls back silently looks, from the outside, like a device that did not update. The metadata should record that a revert occurred and why, and the application, once it is running again, should be able to read that record and report it. Without this, a fleet of devices can be failing every update while appearing healthy.
Attempt counting is not free
The trial count is a small integer and it is decremented on every reset during the trial. On a device that resets frequently for reasons unrelated to the update, such as a brownout on a weak supply or a watchdog that is too tight, the count expires before the application has had a fair chance, and a perfectly good image is rejected.
The usual response is to make the count large enough to survive spurious resets, which weakens the mechanism: an image that hangs will consume the count over a correspondingly longer period, and the device stays down longer before reverting. A small count leaves the application one full attempt plus a margin wide enough to absorb a brownout, and it assumes the application confirms within the first few seconds of running; a count of two or three satisfies that, and anything larger trades recovery time for tolerance of resets that have nothing to do with the image.
A better response distinguishes reset causes. If the reset status register indicates a watchdog reset or a fault, that is evidence about the image and should consume an attempt. If it indicates a power-on reset or an external reset, it is evidence about the environment and should not. Reading the cause requires the bootloader to read and clear the status register before the application does, which is the ownership question raised in what a bootloader must establish before jumping to an application.
A third complication is that the metadata write that decrements the count itself consumes flash. A trial that resets repeatedly writes a record per reset, and the log fills. The log wrap logic has to be correct in this path or the device loses its metadata exactly when it is trying to recover.
When rollback is the wrong answer
Automatic rollback assumes the previous image is better than the new one. That assumption fails in one important case: when the previous image is the reason for the update. A device updated to fix a security flaw, or to correct a calibration that damages hardware, should not revert to the flawed version merely because the new one fails to confirm.
The mechanism has no way to know this. It compares a trial against a confirmed predecessor and prefers the confirmed one. Expressing “this update must not be rolled back” requires either disabling the trial for that update, which gives up all protection, or encoding a policy in the metadata that the bootloader consults before reverting. The policy is a single bit in most designs, and it means the device can end up in a state with no bootable image, which is worse than the flaw it was trying to fix. That is a decision for the product, not the bootloader, and the bootloader’s job is to make it expressible.
A related case is a device whose previous image was never confirmed either. If the first image ever installed is on trial and fails, there is nothing to revert to. The bootloader’s fallback is then a recovery mode: a minimal program that exposes an update interface over whatever link is available, or a ROM bootloader on parts that provide one. Recovery mode is the true last resort and it is the reason the bootloader itself is placed in a region that updates never touch.
References
- MCUboot documentation: Design — MCUboot project
- U-Boot documentation: Boot Count Limit — Das U-Boot project
- ARMv7-M Architecture Reference Manual (ARM DDI 0403) — Arm