Interposing on malloc: How Allocation Tracing Hooks a Heap

Every allocation profiler has to answer the same first question: where does it get its events from? The options are to modify the program, to modify the allocator, or to sit between the program and the allocator without either being aware of it. The third option is the one that gets used, because it works on binaries that cannot be rebuilt and on allocators that cannot be patched. It is also the option with the most subtle failure modes, all of which follow from one rule: the code that intercepts allocation must not itself allocate.

Interposition as a linker feature

The mechanism is symbol interposition, and it comes from the dynamic linker rather than from the allocator. In an ELF shared-object system, a call from one object to a function in another is resolved at load time by searching the global symbol scope in a defined order. The search order is the executable first, then the objects it lists as dependencies in breadth-first order, with the preloaded objects inserted ahead of the program’s own dependencies. The first definition found wins, and every reference to that symbol binds to it.

A replacement allocator therefore needs no cooperation from the program. Compile a shared object that defines malloc, free, calloc, and realloc with the same signatures, place it first in the search order, and every call in the process binds to the replacement. The original definitions remain present in the C library; they are simply no longer reachable by name from the interposed scope.

LD_PRELOAD is the standard way to insert an object at the front of the search order for a single invocation. It is a list of shared objects loaded before the program’s dependencies, and it applies to the program and to every object it loads, including libraries loaded later through dlopen: those inherit the same global scope. This is why a single preloaded object captures allocations made deep inside a graphics library the program never mentions.

The catch in “the first definition found wins” is that the replacement must still be able to reach the original. It cannot call malloc, because that name now refers to itself. Two routes exist:

• dlsym with a handle for the next object in the scope. The dynamic linker can look up a symbol starting after the current object, which yields the C library’s definition. The call needs a handle that represents the scope below the interposer; RTLD_NEXT is the conventional value for exactly this purpose.
• Weak alias binding at load time. Declare the original as a weak symbol in the replacement and bind it once during initialisation. This avoids a dlsym call per allocation but requires the loader to have finished resolving, which is not guaranteed during the earliest allocations.

The first route has a bootstrap problem worth stating plainly: dlsym itself may allocate, and the first call to it typically does. A replacement that calls dlsym on every allocation recurses immediately. The usual fix is a small static arena used by the interposer for its own bookkeeping and for the initialisation window, with a flag that makes the interposer fall back to that arena until the real functions are resolved. The flag must be per-thread if the program is multi-threaded at initialisation, which it usually is not but sometimes is.

What the replacement has to record

An interposer that only forwards calls is useless; the value is the record it keeps. The minimum record is a map from returned pointer to size, since free is not told the size and realloc is told only the new one. That map is itself a data structure that needs memory, which is why interposers carry their own allocator rather than using the one they are tracing.

A size map has a cost per live allocation, and the cost is the reason some profilers store the size in a header immediately before the returned pointer instead. A header changes the returned address, so it breaks programs that assume the pointer is page-aligned or that compute the base of an allocation by rounding down; it also breaks free implementations that require the original pointer. The map avoids both problems at the price of a lookup on every free and a lock or sharded lock around the map’s mutation.

/* Minimal interposer: record size per pointer, forward to the next definition. */
#define _GNU_SOURCE
#include <dlfcn.h>
#include <stddef.h>

static void *(*real_malloc)(size_t);
static void  (*real_free)(void *);
static void record_size(void *p, size_t n);   /* interposer-owned table of live sizes */
static void forget_size(void *p);
static union {                        /* interposer's own storage during resolution */
    max_align_t align;
    char bytes[4096];
} bootstrap;
static size_t bootstrap_used;
static __thread int resolving;       /* re-entrancy guard, per thread */

static void resolve(void)
{
    resolving = 1;
    real_malloc = dlsym(RTLD_NEXT, "malloc");
    real_free   = dlsym(RTLD_NEXT, "free");
    resolving = 0;
}

void *malloc(size_t n)
{
    if (!real_malloc) {
        if (resolving) {                     /* dlsym itself may allocate */
            size_t need = (n + 15) & ~(size_t)15;   /* keep every slot 16-byte aligned */
            if (bootstrap_used + need > sizeof bootstrap.bytes) return NULL;
            void *p = bootstrap.bytes + bootstrap_used;
            bootstrap_used += need;
            return p;
        }
        resolve();
    }
    void *p = real_malloc(n);
    if (p) record_size(p, n);                /* interposer-owned table, not malloc */
    return p;
}

void free(void *p)
{
    if (!p) return;
    if ((char *)p >= bootstrap.bytes && (char *)p < bootstrap.bytes + sizeof bootstrap.bytes)
        return;                          /* came from the bootstrap buffer, not the heap */
    if (!real_free) resolve();
    forget_size(p);
    real_free(p);
}

The guard variable is the essential part. Without it, a call to dlsym that allocates re-enters malloc, which calls dlsym again, and the process either deadlocks on a recursive lock or overflows the stack. The static buffer covers the bootstrap window: while resolution is in progress the interposer serves small requests from its own array and refuses requests larger than it. The buffer is a bump region that is never reclaimed, and the interposer recognises pointers into it so that free can ignore them. That is acceptable only because the window is short: each thread resolves once, and the requests dlsym makes while resolving are few and small. A resolve that needed more than the array holds would fail, and the NULL returned here propagates to the caller rather than being retried from the heap.

Allocations that never reach the hook

Interposition captures calls that go through the dynamic symbol table. It does not capture everything, and the gaps are systematic enough to mislead an investigation.

Statically linked programs. If the program’s allocation calls are bound at link time to a static C library, there is no dynamic lookup and no interposition point. The same applies to a program linked with a static allocator for other reasons.

Internal calls inside the C library. A library that calls its own malloc through a hidden alias, or through a direct internal reference, bypasses the global symbol. glibc historically routes many of its own internal allocations through internal symbols precisely so that replacing malloc does not break its own data structures. Allocations from stdio buffers, locale data, and NSS modules can therefore be missing from a trace while remaining visible in RSS.

mmap and brk used directly. A program or library that maps memory itself never touches the allocator. Region allocators, thread stacks, JIT code buffers, and most language runtimes’ large-object paths fall in this category. A trace of malloc calls can show a stable live set while the process grows steadily through direct mappings.

Allocations before the interposer initialises. Any allocation performed by the loader or by a constructor that runs before the interposer’s own constructor is either missed or, worse, recorded into an uninitialised table. The standard mitigation is a constructor with high priority plus the bootstrap arena, and even then the earliest allocations are attributed to whatever fallback was in place.

Calls through a private allocator interface. A library that exposes its own xmalloc and routes every allocation through it is invisible unless xmalloc itself is interposed. This is common in large C projects and in language runtimes, which maintain their own heap and call the system allocator in large blocks.

The last gap is the one that matters most for footprint work: a runtime that requests memory in 64 MiB arenas and suballocates internally shows almost no allocation traffic at the interposition layer. Tracing it requires hooking the runtime’s allocator, which is a different problem with a different interface, and the per-object overhead accounting that makes such heaps measurable is separate from the interposition mechanism.

Linker-level wrapping instead

A build-time alternative exists for programs that can be relinked. The linker’s wrap option redirects references to a named symbol: every undefined reference to malloc resolves to __wrap_malloc, and the original remains available as __real_malloc. This has two advantages over preloading. It works for statically linked binaries, and it captures calls from objects that were compiled with the wrap in effect regardless of their position in the dynamic scope.

The disadvantage is scope. Wrapping applies at link time to the objects being linked, so a call from a prebuilt shared library that is not relinked still binds to the C library directly. Mixing wrapped and unwrapped objects in one process produces a trace that covers part of the allocation traffic with no indication of which part. For a self-contained binary this is fine; for a program with plugin libraries it is a source of silent undercounting.

Choosing the layer

The three approaches differ in what they can see rather than in how well they see it. Preloading sees every dynamic call in the process, including calls from libraries the program does not know about, and requires no rebuild. Link-time wrapping sees calls from relinked objects only, works without a dynamic linker, and redirects every undefined reference to the named symbol in the objects being linked, wherever that reference sits in the program. Source-level hooks, whether a macro layer or an explicit allocator interface, see exactly the calls the programmer routed through them and nothing else, which is the smallest and most predictable set.

For a first measurement on an unfamiliar binary, preloading is the right default. For a program whose own libraries already route allocation through an internal interface, neither linker approach reaches the allocations that matter, and the work has to move to the stack-based attribution that answers where an allocation came from rather than whether the hook saw it.