Observability · Profiling in Production
Frame Pointers, DWARF Call Frame Information, and What Breaks When Both Are Absent
Two ways to recover a call stack from a captured instruction pointer, the register and table costs of each, and the failure signatures that make a stack look valid when it is not.
A sample yields one address: the instruction the thread was executing. To say anything useful about it, a profiler needs the chain of return addresses that led there. The hardware does not store that chain as a list; it stores it as a sequence of stack frames whose layout is a convention, not an enforced invariant. Recovering the chain means interpreting that convention, and the interpretation is where unwinders succeed quietly or fail silently.
Two mechanisms dominate. One walks a linked list of saved frame pointers. The other consults per-function tables that describe, for every instruction address, how to compute the caller’s frame. They differ in what they cost the compiled program, in what they require at runtime, and in the shape of their errors.
The frame pointer chain and its price
On x86-64 the System V ABI defines %rbp as a general-purpose register with no reserved meaning, but the conventional function prologue saves the caller’s %rbp and then sets %rbp to the current stack pointer:
push %rbp # caller's frame pointer at (%rsp)
mov %rsp, %rbp # this frame begins here
sub $0x20, %rsp # local space below the frame pointer
After the prologue, (%rbp) holds the previous frame pointer and 8(%rbp) holds the return address. Unwinding is then a pointer chase: read the return address, follow the saved frame pointer, repeat. Each step costs two loads and an addition, and there is no table lookup, no metadata, and no dependency on debug information being present at runtime.
The cost is a register. %rbp is one of sixteen general-purpose registers on x86-64, and the compiler would otherwise use it for a local variable, a loop bound, or a spilled value. Committing it to frame bookkeeping reduces the register pool by roughly six percent and adds a push, a move, and a pop to every non-leaf function. Compilers therefore default to omitting the frame pointer at optimization levels that matter, which is what -fomit-frame-pointer requests explicitly and what -O2 on several compilers implies.
The chain has correctness conditions that are easy to violate and impossible to detect from the data alone. The walker must know where the chain ends. A saved frame pointer is not tagged; it is just a value. A walker that follows it past the outermost frame reads whatever the stack happens to contain, and a walker that follows a stale value inside a function that has reused the register for something else produces a plausible address from an unrelated frame. Robust implementations bound the walk by the stack region, require each successive frame pointer to be greater than the previous one on a downward-growing stack, and require 16-byte alignment, since the ABI keeps %rsp aligned at call boundaries. Those checks reject garbage; they cannot detect a chain that is internally consistent and wrong.
Hand-written assembly, just-in-time compiled code, and functions whose prologue is split across basic blocks all break the convention locally. A single frame in a chain that does not maintain %rbp truncates the walk at that point, so the profile shows the leaf and its immediate caller and then nothing.
Table-driven unwinding and what the tables encode
The alternative is to describe the stack layout rather than link it. DWARF call frame information does exactly that: for every address range in a function, it states the rule for computing the Canonical Frame Address, which is the value %rsp had at the call site in the caller, and the rule for recovering each callee-saved register.
The rules are a small bytecode. DW_CFA_def_cfa names a register and an offset, so the CFA is that register plus the offset. DW_CFA_offset says that a saved register lives at a fixed displacement from the CFA. DW_CFA_advance_loc moves the current location forward, which is how a single function’s table can describe a prologue that pushes three registers, an epilogue that pops them, and a body in which the layout is stable. Rules persist until changed, so a table is a sequence of deltas over a large address range rather than an entry per instruction.
At runtime the unwinder searches for the FDE covering the current address, evaluates the rules to compute the CFA, reads the return address from the CFA-relative slot the rules specify, and repeats with the caller’s address and the caller’s CFA as the new stack pointer. On ELF systems the tables live in .eh_frame, which is loadable and, unlike .debug_frame, is present in ordinary binaries because it also serves C++ exception handling.
Lookup cost is the trade. A binary-search index (.eh_frame_hdr) maps an address to an FDE in $O(\log n)$ steps over the number of FDEs, and each frame costs one index lookup, several rule evaluations, and one or more memory reads. Unwinding a thirty-frame stack therefore costs thirty searches and thirty dependent loads, where a frame-pointer walk costs sixty loads with no search. In exchange, the compiled code keeps %rbp for its own use, and the tables work for code that never maintains a chain.
Table size is the other cost, and it is proportional to code size rather than to function count, since prologues and epilogues need separate rules. A rough rule of thumb is a few percent of the text section for C code and more for C++ with heavy inlining, because inlining merges many logical functions into one address range with a longer rule sequence.
Where each mechanism fails
Frame-pointer walking fails in four recognizable ways. The chain terminates early when a frame omits the prologue, which shows up as a stack whose depth varies wildly across samples of the same workload. It produces a wrong caller when a function restores %rbp before its last use of the stack, a pattern compilers emit in tail-position epilogues. It loops when a corrupted frame pointer points at itself, which a monotonic-increase check catches only if the comparison is strict. And it reports a stack of depth one for leaf functions compiled without a frame pointer, which is indistinguishable from a genuine top-level call.
Table-driven unwinding fails differently. The FDE search fails outright for code with no tables: JIT output, hand-written assembly without .cfi directives, and binaries from which .eh_frame has been stripped along with other sections. Rule evaluation fails when it needs a register whose value the unwinder does not have. This is the fundamental limitation: DWARF rules can say “the CFA is %rbp plus 16” only if the unwinder knows %rbp. In a signal handler, the interrupted register state is available from ucontext_t. In a stack walk performed by a separate thread that has attached to a target, the registers must be fetched first, which is why ptrace-based unwinders read the register set before the first frame. When unwinding a chain of thirty frames, only the first frame’s register values are known; every later frame’s registers must be reconstructed from the rules themselves, and a rule that refers to a register the previous step did not restore leaves the unwinder with no value to use.
Tail calls break both mechanisms in the same direction. When a function jumps to its callee instead of calling it, the callee returns directly to the original caller. The frame chain never had the intermediate frame, so the profile is missing a logical level, and no amount of unwinding recovers it because the information was never on the stack.
Hybrid tables and the kernel’s choice
The Linux kernel unwinder does not use either mechanism directly. Its ORC format is generated at build time by a tool that reads the assembler output, and it stores, per address range, the stack pointer offset and the location of the return address as compact integers. The result is table-driven like DWARF but an order of magnitude smaller and cheap enough to consult inside the kernel, where a page fault during unwinding is not an option. The design point is that the tables are generated rather than hand-written, so they cannot drift from the code they describe.
The same principle appears in userspace profilers that synthesize unwind information for JIT code: the runtime emits a table entry whenever it emits a function, which restores the invariant that tables match code. A JIT that does not do this produces stacks that stop at the boundary of compiled code, and the profiler reports the interpreter or the runtime’s own frames in place of the program’s.
What a profiler should report about its own failures
Because every failure mode above yields a stack rather than an error, the useful engineering response is to count the failures. A profiler that records how many samples terminated in an unknown frame, how many frames were recovered per sample, and how many addresses had no covering FDE turns a silent bias into a number that can be compared across builds.
That accounting matters more than the choice of mechanism. A frame-pointer profile of a binary built with frame pointers is exact for the frames it recovers and cheap. A DWARF-unwound profile of a stripped binary with .eh_frame present recovers deeper stacks and costs more per sample. Both are useful; neither is useful when the depth distribution is unknown, because a profile that is systematically shallow over-weights the frames near the top and under-weights everything that was cut off. The next step in the chain, turning those recovered addresses into names, is covered in Build IDs, Separate Debug Files, and Symbol Resolution at Scale.
References
- DWARF Debugging Information Format, Version 5 — DWARF Standards Committee
- GCC Command-Line Options — Options That Control Optimization — GNU Project
- System V Application Binary Interface: AMD64 Architecture Processor Supplement — x86-64 psABI project