Boot Metadata Persistence and the Commit Record

Everything the bootloader decides is derived from a small amount of state: which slot to boot, which slot is confirmed, whether a trial is in progress, how many attempts remain, and what version was last accepted. That state changes on events that are not under software control, namely a power loss, a watchdog reset, or a brownout, and it must be readable after each of them. The commit record is the structure that makes this possible, and its design is a study in what a storage medium can and cannot promise.

A single copy is not a record

Writing a structure to a fixed flash address and reading it back is the obvious implementation and it fails at the first power loss. The failure is not that the write is slow; it is that the write consists of an erase followed by a program, and both are multi-step operations on the device side.

Consider a 64-byte record in a 4 KiB sector. To change one field, the sector must be erased, which sets all 4 KiB to ones, and then the record reprogrammed. Between the erase and the program, the sector holds no record at all. If power fails in that window, the bootloader reads a sector of ones and has no state, and the state it needs to recover includes which slot to boot, which means it cannot proceed.

The window is not small. A sector erase on a typical NOR part takes tens of milliseconds and the device is busy for all of it. The probability of a power loss in that window per event is low, but a device that changes its metadata on every boot performs the operation thousands of times over its life, and the cumulative probability is not negligible. More importantly, the consequence is total: the device has no valid state and no way to reconstruct it.

Two copies in two sectors reduce the probability but do not remove it. A power loss during the erase of the first sector leaves the second intact, so the state survives. A power loss during the erase of the second leaves the first intact. The failure case is a design that erases both before programming either, which reintroduces the window; the copies must be updated in sequence, never together.

Append instead of overwrite

The structure that removes the window entirely is a log. Rather than modifying a record in place, each state change appends a new record to the end of a region. The old record is untouched and remains valid; the new record is either complete and valid or incomplete and invalid. Reading means scanning for the newest valid record. The same construction underlies the slot selection state in dual-bank slots and how a bootloader picks one, where a torn write would otherwise leave the device unable to decide which image to boot.

The cost is that the region fills, and filling requires an erase. When the log reaches the end of its sector, the current state must be written to a different sector before the full one is erased. That gives a two-phase rotation: the new sector receives the state, and only after it is verified does the old sector get erased. At no instant is there no valid record on the device.

The rotation also has to be careful about ordering. If the new sector is erased and written while the old sector is still valid, the device briefly has two valid records with different sequence numbers, and the reader must pick the newer. That is why the sequence number is a monotonic counter rather than a timestamp: a timestamp requires a real-time clock that may not exist, may be wrong after a battery failure, and can repeat if the clock is set backwards. A counter increments on every record and is compared numerically.

Sequence number wraparound is the edge case that a counter introduces and a timestamp does not. A 32-bit counter at one record per boot, with 128 records per sector and a rotation each time, increments once per record and wraps after about four billion records. At one record per second that is over a century, so wraparound is not a practical concern for 32 bits. A 16-bit counter would wrap in under a day of frequent writes, and the comparison must then be modular rather than absolute, which is a source of subtle bugs.

/* Two-sector log. Reading picks the newest valid record across both;
   writing appends to the sector holding the newest, rotating when full. */
#define META_SECTORS 2
#define RECORDS_PER_SECTOR (META_SECTOR_SIZE / sizeof(struct meta_record))

struct meta_record {
    uint32_t seq;
    uint8_t  active_slot;
    uint8_t  confirmed_slot;
    uint8_t  trial_slot;
    uint8_t  trial_count;
    uint32_t version;
    uint32_t crc;              /* CRC-32 over all preceding fields */
};

static int meta_newest(struct meta_record *out)
{
    int found = 0;
    for (int s = 0; s < META_SECTORS; s++) {
        for (unsigned i = 0; i < RECORDS_PER_SECTOR; i++) {
            struct meta_record r;
            flash_read(META_BASE + s * META_SECTOR_SIZE + i * sizeof r,
                       &r, sizeof r);
            if (r.seq == 0xFFFFFFFFu) break;     /* erased: rest is empty */
            if (crc32(&r, offsetof(struct meta_record, crc)) != r.crc) continue;
            if (!found || r.seq > out->seq) { *out = r; found = 1; }
        }
    }
    return found;
}

The break on an erased sequence number is what makes the scan bounded. Without it, the loop reads the entire sector even when the log is nearly empty, and worse, it treats erased records as candidates whose checksum happens to fail. With it, the scan stops at the first unwritten slot, and because records are appended in order, everything beyond is also unwritten.

Checksums, versions, and the fields that must agree

The CRC covers the sequence number and every field used for a decision. A CRC that covers only the payload would accept a record whose sequence number was corrupted into a value that sorts above the real newest, which is the same as accepting a forged record. Including the sequence number in the covered range is not optional.

A record format that changes between bootloader versions needs its own version field, and the version field must be read before the rest of the record is interpreted. The standard arrangement is a fixed-size header with the version and the total record length, followed by a body whose layout depends on the version. A reader that encounters a newer version than it understands should treat the record as unusable rather than misparse it, and should fall back to an older record if one exists. This is what makes it possible to update the bootloader without bricking devices whose metadata was written by the new format.

The CRC also has to cover the length, or a record written short will validate against a truncated body. Since the length is in the header and the CRC is at the end, the reader must locate the end from the header before it can verify anything, which means the header itself is untrusted. The usual resolution is a fixed record size, which removes the problem entirely at the cost of wasted space when fields are added.

How many copies are enough

Three copies in three sectors is the arrangement that survives any single power loss, including one during a rotation, because at every instant at least one sector holds a valid record that is not being modified. Two copies survive a loss during a write but not during a rotation that has already erased the source sector. The difference in flash cost is one sector, which on a part with 4 KiB sectors is 4 KiB, and on a part with 64 KiB sectors is 64 KiB and may be unacceptable.

The number of copies also determines how a device recovers from a sector that has failed. Flash sectors can fail in ways that are not power-loss related: a sector that has exceeded its erase endurance may program and then lose charge, or may fail to erase fully. With three copies, a single failed sector is tolerated by the same logic that tolerates an interrupted write. With two, a failed sector is a lost device.

A related question is where the copies go. Placing them in adjacent sectors means a single erase-block-level failure, or a single physical defect, can take out more than one. Placing them in different erase blocks, and ideally in different banks, reduces the correlation. On a part with a single bank, the physical placement options are limited and the design accepts the risk.

The metadata is not a filesystem

It is worth being explicit about what this structure does not provide, because the temptation to grow it is strong.

It does not provide atomic multi-field updates beyond the record itself. A change that must alter two records is two appends, and a power loss between them leaves a state in which one change is visible and the other is not. The record must therefore be a complete statement of the state, which is why the struct above repeats every field on every append rather than storing deltas.

It does not provide history beyond the log depth. Once a sector is rotated and erased, the records it held are gone. A device that needs to report its last ten update attempts must store them in a separate structure that is not rotated on every boot, or must export them before they are lost.

It does not survive an erase of its own region by other code. An application that writes flash and computes its target address incorrectly can erase the metadata, and there is no defence except making the address calculation a single function used everywhere and marking the region as protected in the memory protection unit where the part supports it. The same caution applies to the commit flag described in erase granularity and power loss, which is only meaningful if the region holding it cannot be rewritten by the code it is meant to constrain.

A bootloader that treats metadata as a small durable key-value store with append semantics and a two-phase rotation gets the properties that matter: the state is never absent, a torn write is detectable, and recovery is a scan rather than a repair. The scan cost is bounded by the sector size, which is the property that makes the whole scheme viable at boot time.