Torn Writes, Journal Checksums, and Detecting a Bad Commit

A journal’s correctness argument rests on one test: does the commit record for this transaction exist on media. If yes, the transaction’s blocks are believed complete and are replayed. If no, the transaction is discarded. The argument assumes that the presence of the commit record implies the presence of everything it commits, and that the blocks it commits contain the bytes that were written.

Neither assumption holds on real hardware. A storage device with a volatile write cache can destage writes out of order, so a commit record can reach media while a payload block earlier in the transaction is still in the cache and is lost when power fails. A write that spans multiple sectors can be split by the interruption, so a block is persisted with its first half from the new version and its second half from the old. And a device that silently drops or corrupts a write, or a bus that flips a bit, produces a block that is present and wrong. A journal that trusts the commit record without verifying the payload replays corruption into the filesystem’s permanent structures.

The failure that the commit test misses

The out-of-order case is the most instructive. Suppose a transaction writes a descriptor, four payload blocks, and a commit record. The device accepts all six writes into its write-back cache and reports completion to the filesystem, which then proceeds to checkpoint by writing the payloads to their final locations. The cache holds all six dirty sectors. Power fails. On restart the device’s cache is gone, but the device had already destaged some of the sectors in an order of its own choosing: the commit record, the descriptor, and two payload blocks survived; two payload blocks did not.

If the filesystem had not checkpointed, recovery would find a valid-looking commit and replay four target blocks of which two hold stale contents. If it had checkpointed, the in-place structures might already be correct and the replay is harmless, but the checkpoint itself could have been interrupted at the same point, leaving two of the four target blocks updated and two not.

The ordering discipline is supposed to prevent this: a barrier between the payload writes and the commit record forces the device to persist the payloads before it accepts the commit. Barriers do exactly that when the device implements them. The remaining failure mode is a device that reports a flush as complete without actually flushing, which is known to occur with drives and with virtualized storage. The filesystem cannot verify that the barrier worked; it can only verify the resulting bytes.

A checksum over the whole transaction

The countermeasure is to make the commit record’s validity depend on the content of what it commits. When a transaction is committed, the filesystem computes a checksum over the descriptor block and all payload blocks, and stores the result in the commit block. Recovery recomputes the checksum from what it reads and compares. A transaction whose payload is incomplete or corrupted fails the check and is discarded, which is the correct action: the transaction never happened, so the filesystem reverts to the state before it.

The checksum changes the recovery contract from “the commit block is present” to “the commit block is present and the transaction it names is byte-for-byte what was written”. The first test is a single-sector read; the second requires reading the whole transaction, which is why checksummed journals pay a cost at mount time proportional to the amount of uncheckpointed log. That cost is bounded by the journal size, so it is a fixed startup penalty rather than one that grows with the filesystem.

The choice of algorithm is constrained by what can be computed cheaply on every block commit. CRC32C, the Castagnoli polynomial reflected with the standard initial and final XOR, is the usual pick because it has hardware acceleration on the common instruction sets and because it detects the error classes that matter here: single-bit flips, burst errors up to the register width, and most multi-bit corruptions. It is not a cryptographic hash and does not need to be; the threat model is random corruption and partial writes, not an adversary. A 32-bit checksum over a 4 KiB block gives a false-accept probability around 2^-32 for random corruption, which is adequate when the alternative is no detection at all.

What the descriptor must also record

Checksumming the payload is necessary but not sufficient, because the descriptor is what maps payload blocks to their destinations. A descriptor that is itself corrupted can point a valid payload at the wrong target block, and the checksum over the payload will still match if the corruption is in the descriptor’s tag list rather than in the block data. The descriptor must therefore be inside the checksum’s coverage, and its own structure must be self-validating: a tag count that exceeds the descriptor block’s capacity, a target block number outside the filesystem, or a run length that crosses a boundary the format does not allow are all conditions recovery must reject rather than act on.

The sequence number adds a second layer. Because the journal is a ring, the same physical offset holds different transactions at different times. A checksum that covers the sequence number binds a transaction’s payload to a particular generation of the log, so a stale block from a previous wrap cannot be mistaken for a current one even if its bytes happen to satisfy the checksum. Recovery’s scan therefore validates three things per transaction: the descriptor parses, the sequence number is the expected next value, and the checksum over descriptor plus payload matches the value in the commit block.

struct journal_commit {
    uint32_t magic;
    uint32_t blocktype;   /* commit */
    uint32_t sequence;
    uint32_t crc32c;      /* over descriptor + payload, seeded with sequence */
};

/* Validate a candidate transaction in the ring before touching any inode. */
static bool transaction_is_replayable(const struct journal *j, uint64_t off,
                                      uint32_t seq, uint64_t *next_off)
{
    struct journal_commit c;
    uint64_t commit = off + j->desc_blocks + j->payload_blocks;
    if (!read_block(j, commit, &c))
        return false;
    if (c.magic != JOURNAL_MAGIC || c.blocktype != COMMIT || c.sequence != seq)
        return false;

    uint32_t crc = crc32c_seed(seq);          /* bind to this generation of the log */
    crc = crc32c_update(crc, j->desc, j->desc_bytes);
    for (uint32_t i = 0; i < j->payload_blocks; i++) {
        if (!read_block(j, off + j->desc_blocks + i, j->scratch))
            return false;
        crc = crc32c_update(crc, j->scratch, j->block_size);
    }
    if (crc != c.crc32c)
        return false;                          /* incomplete or corrupt: discard */
    *next_off = off + j->desc_blocks + j->payload_blocks + 1;
    return true;
}

The loop reads every payload block to compute the checksum, which is the cost of the stronger test. Implementations avoid re-reading by computing the checksum incrementally as blocks are written and keeping the partial state in memory until the commit, so the write path pays a pass over data already in cache and the read path pays a pass over the log at mount. Neither is proportional to filesystem size.

Where the guarantee still stops

A checksummed journal detects corruption in the log, not corruption in the filesystem after replay. Once a payload block is written to its home location, the journal has done its job; a bit that flips in that block a month later is the concern of the filesystem’s own data and metadata checksums, which is a separate mechanism with a different scope. The journal’s checksum protects the window between the write of a transaction and the point at which checkpointing retires it, which is exactly the window in which the ordering discipline is the only other defence.

There is also a case the checksum cannot fix: a transaction that was correctly written and committed, but whose payload was computed by the filesystem from state that was itself inconsistent. A journal faithfully replays a transaction that records a wrong pointer, and no checksum can tell. This is why journaling is described as providing structural consistency rather than semantic correctness: it guarantees that recovery produces a state the filesystem’s own invariants accept, not that the state is the one the application intended. The mechanisms that avoid this class of error entirely work by never modifying live structures, which is the copy-on-write argument, and the mechanisms that detect it after the fact rely on checksums stored in each metadata block rather than in the log.

The residual risk after all of this is a device that lies about flush completion and a filesystem that does not checksum. That combination is not hypothetical; it is the reason the checksum feature exists and the reason enabling it is recommended on any volume whose storage stack includes a write cache the filesystem does not control. A journal without checksums is a performance choice with a specific, enumerable failure mode attached.