Parsing Untrusted Binaries Without Trusting Their Length Fields

Every field in an executable is an integer read from a file that may not be the file its author intended. An ELF header claims a program header table at some offset with some entry count, as described in ELF Program Headers and the Mapping a Loader Performs Before _start; a PE section header claims a raw data pointer and a size, in the layout covered by PE/COFF Relative Virtual Addresses and the Import Directory; a DWARF abbreviation claims an attribute list of unbounded length. In a trusted build these claims are true by construction. In a file received from elsewhere, each claim is a hypothesis, and the parser is the code that decides whether to act on it.

The failure mode is uniform across formats. A parser reads a count, loops that many times reading a fixed-size structure, and advances a pointer without checking whether the structure lies inside the buffer. A count of 0xffffffff in a file of 400 bytes produces either a wild read or an arithmetic overflow that wraps into a small number. Both outcomes have been the basis of real memory-safety defects in loaders, disassemblers, and file inspectors, which is why the discipline of parsing binary formats is mostly the discipline of making each claim cheap to verify and impossible to forget.

Three sources of trouble: counts, offsets, and arithmetic

The first category is the count that does not match the data. ELF’s e_phnum and e_shnum, PE’s NumberOfSections, Mach-O’s ncmds, and DWARF’s abbreviation lists all specify a length that the parser uses to control iteration. A count larger than the available bytes is not a malformed value the parser can ignore; it is a claim that must be compared against the remaining buffer before the first iteration, not inside the loop where the comparison happens too late for the count itself.

The second category is the offset that points anywhere. ELF’s e_shoff, PE’s PointerToRawData, Mach-O’s fileoff, and every RVA in a PE data directory are positions that may lie outside the file, inside a different section, or inside the header itself. The last case is the interesting one: a section header table that overlaps the program headers is structurally possible in the format and produces a parser that reads its own metadata as section descriptors. There is no rule in ELF forbidding it, so a reader that cares about self-consistency must check for overlap explicitly.

The third category is arithmetic that overflows before it is checked. The canonical pattern is if (offset + size > len) error; followed by a read of size bytes at offset. When offset and size are both near the maximum of their type, the sum wraps to a small value, the check passes, and the read goes out of bounds. The defect is invisible in review because the check looks correct; the fix is to compare against the remaining length rather than the total, or to compute in a wider type.

/* A cursor that owns the bounds check, so no caller can forget it. */
#include <stdbool.h>
#include <stdint.h>
#include <string.h>

struct cursor { const uint8_t *base; size_t len; size_t pos; };

static bool take(struct cursor *c, size_t n, const uint8_t **out) {
    if (n > c->len - c->pos) return false;   /* subtraction cannot wrap */
    *out = c->base + c->pos;
    c->pos += n;
    return true;
}

static bool u32le(struct cursor *c, uint32_t *v) {
    const uint8_t *p;
    if (!take(c, 4, &p)) return false;
    *v = (uint32_t)p[0] | (uint32_t)p[1] << 8 |
         (uint32_t)p[2] << 16 | (uint32_t)p[3] << 24;
    return true;
}

The subtraction in take is the whole point. Comparing n > c->len - c->pos is safe because pos is an invariant of the cursor: it never exceeds len, so the subtraction cannot underflow. Comparing c->pos + n > c->len is unsafe for the reason above. The invariant is maintained by the only function that mutates pos, which means a caller cannot violate it without editing the cursor’s implementation.

Two details make the cursor more than a convenience. It holds the buffer’s length as a field rather than passing it alongside every pointer, which removes the possibility of passing a length that belongs to a different buffer. And take returns a pointer into the buffer rather than copying, so parsing large structures does not duplicate them, at the cost that the returned pointer’s lifetime is tied to the buffer. For a read-only inspector that is the right trade; for a parser that retains structures after the buffer is freed, it is not.

Validating a table before iterating it

A cursor makes each individual read safe. It does not make a loop safe, because the loop’s iteration count comes from the file. The pattern that handles this is to validate the table’s extent once, then construct a bounded view over it.

For ELF’s section header table the computation is: the table occupies e_shnum * e_shentsize bytes at e_shoff. Both factors are attacker-controlled 16-bit values, so the product fits in 32 bits and cannot overflow a size_t, but it can easily exceed the file. The check is that e_shoff plus the product is within the buffer, performed in a type wide enough to hold the sum. After that, each entry is read at e_shoff + i * e_shentsize, and the stride is e_shentsize rather than a constant, so an entry size smaller than the structure’s real size is itself a malformed condition that must be rejected before the first read.

The analogous check for Mach-O is that the sum of cmdsize values equals sizeofcmds and that no individual command’s size is below the minimum header size. Without the per-command check, a command claiming size zero produces an infinite loop that never consumes the remaining bytes. Without the sum check, a command stream that ends early leaves the parser reading into whatever follows the load commands. The structure of that stream is described in Mach-O Load Commands and the Segments a dyld Image Is Built From.

DWARF is harder because entries have no length field at all. The extent of a compilation unit is given by its header’s length, but the extent of one entry is the sum of its attributes’ encoded sizes, which requires decoding every attribute in order with the form from the abbreviation. There is no way to skip an entry without understanding it, and no way to know the entry is malformed until an attribute decode runs off the end of the unit. The defence is to bound each decode by the unit’s end rather than by the section’s end, so that a unit cannot read into its neighbour, and to bound the total entry count, since a unit can contain an arbitrary number of zero-size entries. The abbreviation encoding that makes this necessary is set out in DWARF Debug Information: Abbreviation Tables and the Line Number State Machine.

The checks the formats do not perform

Some invariants that a reader expects are not stated by any specification, and a parser that relies on them inherits the assumption. The most consequential are ordering, disjointness, and monotonicity.

Ordering: ELF does not require that program headers appear in ascending address order, nor that section headers appear in any order. A loader that merges adjacent segments while walking the table in file order produces wrong mappings for a file that lists them out of order. The correct approach is to collect all entries and sort them by address before merging, not to assume the file did it.

Disjointness: two PT_LOAD segments may overlap in virtual address, and the format does not forbid it. A loader must decide what an overlap means, and the decision affects security: overlapping segments with different permissions can be used to make a page appear read-only while a writable alias exists. A reader that reconstructs an image in memory must either reject overlaps or resolve them with a defined precedence.

Monotonicity: DWARF line tables do not require that addresses increase within a sequence, though they normally do, and they explicitly allow the address to decrease when a new sequence begins. A reader that assumes monotonicity to binary-search a line table produces wrong results for functions placed out of order, which happens routinely when a linker groups cold code separately.

Version handling is a fourth category. Every one of these formats has multiple versions, and the differences are not cosmetic. DWARF 5 changed the line table header layout, so a reader that assumes the version 4 header misparses every version 5 file. PE32 and PE32+ differ in field widths, and the optional header’s magic is the only reliable discriminator. Mach-O’s 32-bit and 64-bit headers differ by a magic value and by four bytes of reserved field. A parser should read the version, select a decoder, and fail closed on an unknown value rather than attempting the nearest known layout.

Fuzzing as the only convincing evidence

The property that matters is not “this parser handles the malformed files anyone thought of” but “this parser does not read outside its buffer for any input”. Testing by example cannot establish the second property, because the space of malformed files is larger than any set of examples. Coverage-guided fuzzing explores that space by mutating a corpus of valid files and retaining inputs that reach new code paths, which is the closest available approximation to exhaustive testing of a parser.

The practical requirements are specific. The parser must be reachable from a harness that takes a byte buffer and returns without side effects. The build must include instrumentation so that the fuzzer can observe coverage. The parser must be deterministic: no dependence on the clock, the environment, or the file’s own name, since the fuzzer will vary all three. And sanitizers must be enabled, because a read one byte past the end of a heap allocation is usually invisible without one.

A parser that passes a fuzzing campaign has demonstrated the absence of the defects the fuzzer can reach, not correctness. Two limitations are worth stating. A fuzzer that only mutates headers will not reach deep into a DWARF expression evaluator that is only invoked for specific attribute forms, so the corpus needs seeds that exercise each format’s optional features. And a fuzzer cannot detect a semantic error that is memory-safe: a parser that reads a plausible but wrong section header and reports a symbol at the wrong address passes every sanitizer.

Structured failure instead of defensive silence

The alternative to bounds checking everywhere is a design in which every read can fail and failure is propagated to a single exit. That is what the cursor achieves: take returns false, the caller returns false, and the top-level function reports “malformed file” with the offset at which parsing stopped. No read is unchecked, and no error path is silent.

The reporting detail matters more than it appears. A diagnostic that names the failing offset and the structure being decoded turns a fuzzer crash into a reproducible test case and turns a user’s report of a rejected file into a specific malformed field. A parser that returns a generic failure discards the information that would let someone determine whether the file is corrupt or merely unusual, and the difference between those two is often the whole question.

There is a cost to the discipline. Every accessor returns a status, every structure decode has an error path, and the code grows in proportion to the number of fields rather than to the number of decisions. The compensation is that the growth is mechanical and reviewable: a reader can check that every access goes through the cursor and that the cursor’s invariant holds, which is a much smaller thing to verify than the correctness of a hundred hand-written bounds checks.

The formats do not help. None of the three carries a checksum over its own structure, and the one that carries a signature, Mach-O, is signed by a mechanism that is itself optional in the format. A parser is therefore the only integrity check that exists for most inputs, and the only assumption it can safely make is that every field it reads is a claim rather than a fact.