Debug Information for Optimized Code: Line Tables, Location Lists, and Lost Variables

Debug information produced for unoptimized code is simple enough to be described by a stack discipline: each variable has a frame offset, and the debugger reads memory at that offset. Optimized code destroys that correspondence. A local may live in a register for part of its lifetime and in a stack slot for another part, may be recomputed rather than stored, may be shared with another variable, and may have no storage at all after its last use. A debugger that reports the value of such a variable is doing reconstruction, not lookup, and the description it reconstructs from is more elaborate than the unoptimized case.

The design constraint is that the compiler must describe its own transformations well enough for a third-party debugger to undo them, without the debugger knowing which optimizations ran. The DWARF format solves this with a small set of constructs that generalise over the transformations: location lists for values that move, inline scopes for code that moved, and a formal statement of what a location expression computes.

What changes when the compiler is allowed to optimize

Three transformations account for most of the difficulty.

Register allocation removes the memory home of a variable. A local that was a stack slot becomes a register, and the register is reused for other variables at other times. The debugger must know which register, during which range of instruction addresses, and it must know when that range ends.

Dead code elimination removes the variable entirely. After its last use, a variable has no value because there is nothing to hold one. Reporting a stale value from a reused stack slot would be worse than reporting nothing, so the debug information must mark the location as unavailable rather than pointing at memory that now holds something else.

Inlining moves code. A function body is spliced into its caller, so a line table that maps instruction addresses to source lines must map them to lines in the inlined function, with a record of the call site that brought them there. Without that record, stepping through the caller would appear to jump into another file at random.

Each of these has a corresponding DWARF mechanism. The important point is that the mechanisms are general: they describe ranges and expressions, not specific optimizations, so a new optimization that produces the same kinds of effects needs no new debug format.

Location lists: a variable’s home as a function of address

A location list is a sequence of entries, each covering a contiguous range of instruction addresses and giving a location expression to evaluate within that range. An entry may say the value is in a named register, in a frame-relative memory slot, at a fixed address, or in a piecewise combination. An entry may also say the value is unavailable, which is the correct answer after the variable’s last use.

The debugger’s lookup is a binary search over the ranges. That makes the cost of resolving a variable logarithmic in the number of entries, and the number of entries is typically small, since a variable changes home only where the compiler chose to move it. The lists are stored in a separate section and referenced by offset from the variable’s description, which keeps the description compact when a variable is optimized away entirely and needs no list at all.

Two details affect what the debugger can print. The first is that a location expression can be arbitrarily complex: it is a small stack machine, and it can compute an address from registers and constants, dereference it, and apply arithmetic. A variable whose address is recomputed from the stack pointer after every call is described by an expression rather than a fixed offset, which is why the debugger can still find it when the frame layout changes mid-function. The second is that a variable can be split into pieces, each with its own location, which is what happens when a struct’s fields are allocated to different registers. The debugger must assemble the pieces to present a coherent value, and a piece that is unavailable makes the whole value partly unknown.

Inline scopes and the line table’s two coordinate systems

The line table maps an instruction address to a file and line. With inlining, that mapping is ambiguous: an instruction may correspond to a line in the caller and, simultaneously, to a line in the callee. DWARF resolves the ambiguity with a stack of inline scopes recorded as debugging information entries, each naming a function and the call site that brought its code in. The line table itself keeps one mapping per address; the chain of inline scopes is what says which of those scopes an address belongs to. A line table entry can carry a discriminator, but its purpose is narrower, distinguishing several entries that describe the same source line.

The practical consequence for a debugger is that it maintains two notions of position: the source line at which execution is stopped, and the sequence of inlined frames that produced it. Stepping operations act on the second, because a step into an inlined call should stop inside the inlined body even though no call instruction exists. A step over should advance to the next line in the caller, skipping the inlined body’s lines even though they are interleaved in the address space.

This is where debuggers differ most visibly. A debugger that ignores inline scopes reports a stop in a function that was never called, at a line that does not exist in the current file. A debugger that handles them reconstructs the logical call stack from the inline records and the actual call stack from the frame pointers, and presents a merged view. The merge is not always unambiguous: tail calls and sibling calls destroy the correspondence between the logical stack and the physical one, because the caller’s frame has been reused. The debug information can record that a call was a tail call, and the debugger can report a frame as reconstructed rather than actual, but the reconstruction is limited by what the compiler recorded.

Why a variable can be printed as optimized out

The message a debugger prints when a variable has no location is accurate: the value is not available at the current instruction address. The interesting question is why the compiler did not keep it.

Keeping every variable live for debugging is possible and has a cost. The compiler would have to spill values to memory at every point where a debugger might stop, which adds instructions and memory traffic and prevents optimizations that depend on values staying in registers. The alternative is to track debug uses separately and only keep values that a debugger could ask for at a point where the debugger can actually stop. Optimizing compilers take the second route, and it is why the reported availability depends on where execution is stopped: a variable may be describable at one instruction address and not at the next, because the compiler’s decision to keep its value in a stable location applies to a specific range of addresses and no more.

A variable that is used only to compute something else, and whose value is therefore recomputed rather than stored, may still be printable. If the compiler records the expression that computes it from values that are available, the debugger evaluates that expression. This is the same mechanism as a location list, applied to a value that has no home at all: the location expression is a computation, not an address. It is also the reason a variable can appear correct at one point and unavailable a few instructions later, since the values the expression depends on may themselves have been overwritten.

int sum(const int *v, int n) {
    int total = 0;
    for (int i = 0; i < n; i++) total += v[i];
    return total;   /* at -O2, total is likely in a register here */
}

In this function the loop counter and the accumulator are both candidates for register allocation, and after the loop the counter’s value is dead. A debugger stopped on the return line will typically show total in a register and i as optimized out, and the second answer is the correct one rather than a failure.

Reading the output when the debugger is the wrong tool

When a debugger’s report is suspect, the debug information itself is inspectable. Tools that dump the sections show the location lists, the inline scope tree, and the line table’s address-to-line mapping, and they answer questions the debugger’s user interface hides: which ranges a variable’s list covers, whether an entry says unavailable, which function an address is attributed to.

The dump is also the way to diagnose a mismatch between compiler and debugger versions. Debug information is a format with multiple versions, and a consumer that does not understand a newer version’s constructs may silently ignore them rather than failing. The symptom is a debugger that works for unoptimized builds and reports nonsense for optimized ones, which is easy to misattribute to the optimizer when it is the format version that changed. The same sections are also what a staged compiler build has to normalise before the binaries from two stages can be compared byte for byte, since the sections embed the compilation directory unless a prefix map rewrites it.

The underlying trade-off does not go away. Debugging optimized code is possible because the compiler records enough to reconstruct the program’s state, and the reconstruction is incomplete exactly where the compiler judged the recording too expensive. Building with optimization disabled removes the reconstruction problem and changes the program’s timing and memory behaviour, which is why the two builds answer different questions. The toolchain’s job is to make the optimized build’s record good enough that the questions can be asked of it. That record is also one of the places where build artifacts acquire nondeterminism, since debug sections embed the producer string, the compilation directory, and sometimes a timestamp.