Chunked Transfer and Retransmission in Firmware Delivery

A firmware image is a few hundred kilobytes to a few megabytes and the link it crosses may be a UART at 115200 baud, a CAN bus, a BLE characteristic with a negotiated MTU, or a USB endpoint. On the slowest of those, a 512 KiB image takes tens of seconds to move even with no errors, and the device is not in a state where it can be restarted arbitrarily. The transfer layer exists to make that movement reliable in the presence of loss, and to make it resumable when the link drops for a minute.

Why the transfer is not just a byte stream

If the link were perfect, an update would be: erase the slot, then write bytes as they arrive. Every property that makes this insufficient is a property of the link or of the flash.

Bytes arrive in arbitrary quantities. A UART delivers one character at a time at a fixed rate, framing each one with start and stop bits but imposing no structure on the stream as a whole; a USB bulk endpoint delivers whatever the host chose to send, which can be any length up to the maximum packet size. A receiver that writes flash as bytes arrive will program a page byte by byte, which is not how flash works, so it must buffer to a page and program once. The buffer is the first state the receiver must maintain.

Errors occur and are not always detected. A UART can be configured with parity, which detects every error that flips an odd number of bits in a character and none of those that flip an even number. A CAN frame has a CRC and a controller that retransmits automatically on error, which makes the bus reliable but not ordered across nodes. A radio link has a checksum at the packet layer and drops packets silently. The receiver therefore needs its own end-to-end check, because per-link checks only cover the link.

The link can disappear. A cable is unplugged, a radio peer moves out of range, a host process is restarted. Without a notion of position, a reconnect has to begin again from zero, and for a 1 MB image over a slow link that is a cost the design may not be able to pay.

Numbering the chunks

The standard answer to all three is to number the data units and require acknowledgement. TFTP, specified in RFC 1350, is the smallest complete example: 512 bytes of data per packet, a 16-bit block number, and one acknowledgement per block. The block number provides the position, the acknowledgement provides the loss signal, and the fixed data size provides the buffering bound.

The choice of 512 bytes is not arbitrary. It is small enough that a receiver can hold one block plus a small amount of state in RAM, and it is large enough that the per-block overhead is a small fraction of the payload. A TFTP data packet is 4 bytes of header plus 512 bytes of payload, so the overhead is under 1 percent. Reducing the block to 128 bytes would raise the overhead to 3 percent and double the number of acknowledgements, which on a link with any round-trip latency is a large throughput loss.

That last point is the throughput equation, and it is worth stating plainly. With one acknowledgement per block, the sender cannot have more than one block in flight, so the throughput is bounded by block size divided by round-trip time, regardless of link bandwidth. On a link with 10 ms round-trip and 512-byte blocks, the ceiling is about 51 KB/s even if the link could carry a megabyte per second. RFC 7440 addresses exactly this by allowing a window of blocks to be sent before an acknowledgement is required, which multiplies the ceiling by the window size. The window is bounded by what the receiver can buffer, so the receiver advertises it.

The window is also bounded by what the receiver can store durably, and this is where the transfer layer meets flash. A receiver that acknowledges a block before writing it has promised something it may not deliver. If power fails after the acknowledgement and before the write, the sender believes those bytes are stored. A correct receiver acknowledges only after the flash operation completes, which makes the acknowledgement latency include a page program time, and that time can dominate the round trip on a slow part.

/* Fixed-size record with a sequence number, as it appears on the wire.
   The sequence number is the position; the length field allows the final
   record to be short. */
struct fw_record {
    uint16_t seq;          /* 0-based, wraps at 65536 */
    uint16_t length;       /* payload bytes, 1..MAX_PAYLOAD */
    uint8_t  payload[MAX_PAYLOAD];
    uint32_t crc;          /* CRC-32 over seq, length, and payload */
};

static int handle_record(const struct fw_record *r, uint32_t *write_pos)
{
    if (r->length == 0 || r->length > MAX_PAYLOAD)
        return -1;
    if (crc32(r, offsetof(struct fw_record, crc)) != r->crc)
        return -1;                     /* corrupt in transit: ask for a resend */
    if (r->seq == (uint16_t)(expected_seq - 1))
        return 2;                      /* duplicate: ack again, but do not rewrite */
    if (r->seq != expected_seq)
        return 0;                      /* gap: ignore, request expected_seq */

    if (flash_write(*write_pos, r->payload, r->length) != 0)
        return -1;                     /* real failure: abort, do not ack */
    *write_pos += r->length;
    expected_seq++;
    return 1;                          /* caller sends the acknowledgement */
}

The duplicate case deserves attention because it is not an error. An acknowledgement lost in the reverse direction causes the sender to retransmit a block the receiver already stored. The receiver must recognise the sequence number as one it has already accepted and acknowledge again without writing, because writing it twice would advance the write position past the block. A receiver that treats a duplicate as a failure will abort transfers that were proceeding correctly.

Where retransmission is decided

Two designs are possible and they differ in which side holds the timer.

Sender-driven retransmission puts a timer on each outstanding block. If no acknowledgement arrives within the timeout, the block is resent. This is what TFTP does, and it requires the sender to be a process that can maintain timers, which is true of a host tool and false of a minimal bootloader. It also requires the timeout to be chosen sensibly: too short and the sender retransmits into a link that is merely slow, adding traffic to a congested link; too long and a loss costs a long stall.

Receiver-driven retransmission has the receiver request a specific block by number when it detects a gap. This suits a device that is already running a protocol loop and can hold a request in a small buffer, and it makes recovery from a single loss cost one round trip rather than one timeout. It requires the receiver to detect the gap, which means sequence numbers must be strictly increasing within a window and the receiver must be able to tell “not yet arrived” from “lost”, which is a timeout on its own side.

In practice the receiver-driven form is what embedded bootloaders implement, and the reason is resource asymmetry: the host has memory and timers, the device has neither. The device’s protocol loop reads a record, checks it, writes it, and acknowledges. If a record does not arrive within its own timeout, it sends a request for the next expected sequence number and the sender responds by resending from there. That is a single mechanism serving both loss and reordering.

The timeout value is a design parameter with a real constraint. It must exceed the worst-case time between records, which includes the sender’s own processing, the link’s latency, and, on a shared bus, the time other nodes hold the medium. A 500 kbit/s CAN bus carries a full frame in roughly a quarter of a millisecond, so a sender that has to wait behind several higher-priority nodes accumulates a delay of a millisecond or more, and a timeout set to a few frame times will produce spurious requests that themselves add traffic. The usual approach is to make the timeout generous and to make the recovery cheap, so that a spurious request costs one duplicate record rather than a restart.

Resume across a disconnect

A transfer that can be interrupted needs a durable notion of how far it got, and the only durable storage is the flash being written. The write position is therefore recoverable from the metadata, not from RAM. Two approaches are common.

The first records the position in a metadata record after every N blocks, where N is chosen so that the record write cost is a small fraction of the data written. If a page is 2 KiB and a record is 32 bytes, recording after every page costs about 1.5 percent in write volume and bounds the rework after a disconnect to one page. The metadata log described in dual-bank slots and boot selection is the natural place for it.

The second derives the position from the image itself. Since the image is written in order and the header is written last, the receiver can scan the slot for the last non-erased page and resume from there. This costs a scan at reconnect, which is bounded by the slot size, and it requires that the erase pattern be distinguishable from data, which is true when the image contains no long runs of 0xFF and false when it does. A bitmap of written pages, stored separately, avoids the ambiguity at the cost of more state.

Neither approach resumes mid-page, because flash cannot be programmed twice. The resume point is a page boundary, and any bytes delivered beyond the last complete page are discarded and re-sent. This is why the transfer layer’s notion of position must be the flash position, not the byte count received: a protocol that resumes at the byte count will try to write a partial page.

What the layer cannot fix

Three problems are outside the transfer layer’s reach and are worth naming so they are not mistaken for protocol bugs.

A link that delivers data faster than flash can absorb it will overflow any fixed buffer. Flow control has to exist at some layer, and if the transport has none, the protocol must provide it by refusing to acknowledge until it has room. This converts a bandwidth problem into a latency problem, which is the correct trade: a stalled sender is recoverable, an overrun buffer is not.

A link whose errors are correlated in time will defeat a fixed timeout. A burst of interference lasting a second will cause several retransmissions in a row, and a sender that gives up after a small number of attempts will abandon a transfer that would have succeeded. The attempt limit has to be a function of the expected outage duration, not of the packet loss rate.

A device that resets during a transfer loses everything not in flash. The protocol state, the expected sequence number, the buffer contents, and the window are all gone, and the resume path has to reconstruct them from flash. That reconstruction is the reason the durable position record is written before the acknowledgement rather than after: the order determines whether a reset can produce a state in which the host believes bytes are stored that the device never wrote. It is also why the resume point is a page boundary rather than a byte offset, since the medium cannot be programmed twice at the same address, as erase granularity and power loss sets out.