Allocator Mismatch and Cross-Runtime Deallocation

Memory returned by malloc is not a portable value. It is an address inside a heap that belongs to a particular allocator instance, and only that instance can interpret the metadata preceding the address or return the block to the correct free list. Calling free on a pointer from a different allocator is undefined behaviour in the strict sense: the standard does not describe what happens, and implementations do not attempt to detect it.

The rule is simple to state, allocate and free with the same allocator, and hard to satisfy across a language boundary, because the boundary is exactly where the identity of the allocator becomes invisible. A pointer is a pointer. Whether it points into a C runtime’s heap, a garbage-collected arena, or a pool owned by a third library is not encoded in its bits.

How a process ends up with more than one heap

On Linux with glibc, the C runtime’s allocator is a single global implementation, and every module that calls malloc through the dynamic symbol table reaches the same instance. That is why a shared library built separately can return memory that the executable frees, and why the pattern works often enough to be assumed reliable.

It stops working in several ordinary situations. A shared object that links the C runtime statically carries its own copy of the allocator, with its own global state and its own sbrk or mmap bookkeeping. On Windows the situation is more common, because the C runtime is versioned and a library built against one version may load a different one; the runtime has an explicit rule that memory allocated in a DLL must be freed in the same DLL, and the documented way to cross the boundary is a pair of exported allocation and deallocation functions rather than malloc and free. A library that returns a pointer to memory it allocated and expects the caller’s free to handle it is violating that rule even when the two runtimes happen to share a heap today.

Managed runtimes add a second layer. A garbage-collected heap does not have a free list that a foreign free could return a block to, and its objects may be moved by a collector that runs on another thread. Handing out a raw pointer to object memory is possible only under a pinning protocol, and even then the memory must be released by unpinning rather than by free. A binding that converts a managed string into a C string by returning a pointer into the object’s storage creates a pointer that becomes invalid at the next collection.

Custom allocators are the third case, and the most common in practice. A library that uses an arena for all its internal allocations returns blocks that are not individually freeable; releasing one is meaningless, and the arena is destroyed as a unit. A library that uses a size-class pool will misbehave if handed a pointer of a class it does not own. None of these is visible from the pointer.

What the allocator needs to know that the caller cannot

A general-purpose allocator stores bookkeeping near the block. The exact layout is implementation-specific, but the common designs place a header immediately before the returned address, or store metadata in a separate table keyed by the block address, or encode the size in the page containing the block. Freeing a foreign pointer means the allocator reads whatever bytes precede the address and treats them as its own header. If those bytes happen to look like a valid header, the allocator links the block into a free list that overlaps live memory; the corruption surfaces later, at an allocation that has nothing to do with the original error.

This is why allocator mismatch produces a delayed and misattributed crash rather than an immediate fault. The immediate fault, when it happens, is the benign case: the allocator notices an invalid size or an unmapped address and aborts. The corrupting case is silent.

Size-class allocators make the problem sharper. Freeing a pointer into the wrong class may push it onto a list whose blocks have a different size, and the next allocation from that list returns a block of the wrong size to a caller that will write past its end. Alignment assumptions compound it: an allocator that returns sixteen-byte-aligned blocks will corrupt its own free lists if given a pointer that is only eight-byte-aligned, which is what happens when a caller frees a pointer to the interior of a block rather than its start.

API shapes that make the identity explicit

There are four shapes in common use, and they differ in where they put the obligation.

The first returns memory allocated by the callee and requires the caller to call a named release function. The release function is part of the interface, exported alongside the allocation, and the documentation states that the platform free is not acceptable. This is the only shape that works across allocators that do not share a heap, and its weakness is that the obligation is a convention rather than a type.

The second accepts a caller-provided buffer. The caller allocates, the callee writes, the caller frees. Nothing about allocation crosses the boundary, so the identity problem cannot arise. The cost is that the caller must know the size, which pushes the interface toward the two-call pattern described in string encoding and buffer ownership.

The third exports the allocator itself. The library provides functions to allocate and free its own blocks, and the caller uses those for any memory the library will later consume. This is the most flexible shape and the most demanding, because it requires the caller to route every allocation through the provided pair and to remember which blocks belong to which allocator.

The fourth avoids per-object allocation entirely by making the object opaque and lifetime-scoped. A handle is created by one call and destroyed by another; the caller never sees the pointer, never frees it, and never learns its size. This is the shape taken by most C APIs that must be usable from many languages, and it is covered in opaque handles and ownership transfer.

/* The library exports both halves of every allocation it performs. */
typedef struct db db;

db   *db_open(const char *path);   /* block comes from the library's heap */
void  db_close(db *h);             /* the only valid way to release it */

char *db_dup_string(const char *s);/* same heap as db_open */
void  db_free(void *p);            /* must be used, not free() */

The fragment is deliberately small because the error is small and invisible to the compiler. db_free exists only because the caller cannot know which allocator produced the block; a binding generator that maps db * to a language-level pointer type will expose the pointer, and nothing in the generated code prevents a caller from passing it to the language’s own deallocator. The exported pair is the interface stating the obligation, and a binding that ignores it is where the corruption begins.

Where the two sides disagree about who owns the result

Functions that return a struct by value create an ownership question that is easy to miss. The struct itself is copied into the caller’s frame, so no allocation crosses the boundary for the struct’s storage. But if the struct contains a pointer to a string, that string was allocated somewhere, and the ownership of the pointee is a separate question from the ownership of the struct. An API that returns { char *name; int code; } by value has transferred a struct and possibly a string, and the two may have different owners.

The convention that resolves this is to state, per field, whether the callee or the caller owns the pointee, and to provide a release function for every field the caller owns. APIs that document this are rare; APIs that document it and enforce it in a generated binding are rarer. The consequence is that most leaks at language boundaries are small, repeated, and attributed to the wrong component, because the caller believes it received a value and the callee believes it transferred an obligation.

A related asymmetry appears with realloc. A callee that grows a buffer may return a new address, leaving the old one freed. A caller holding the old address and the new one simultaneously has a dangling pointer, and a caller that frees the old address after a successful realloc double-frees. Interfaces that resize caller-visible buffers should therefore return the new pointer and require the caller to discard the old one, and the documentation should say so explicitly rather than relying on the caller to infer it from the realloc contract.

Testing for mismatch before it becomes corruption

Allocator mismatch is one of the few FFI defects that can be detected mechanically. Debug allocators that fill freed memory with a pattern and check it on the next allocation will catch a foreign free when the pattern check fails. Tools that intercept the allocator entry points and record the allocating caller’s module will report a free from a different module as an error rather than a corruption. Neither approach proves the interface correct, but both convert a delayed crash into an immediate report naming the two modules involved.

The static defence is more valuable: make the release function part of the interface, name it in the declaration, and generate the binding from that declaration so that the release call is emitted automatically. An interface where the only way to release a resource is to call a function that the binding knows about cannot be misused by forgetting it, which is why the opaque handle shape dominates the design of libraries intended for many languages. The remaining exposure is the pointer that is not a handle, such as the string returned by a lookup or the array returned by an enumeration, and for those the interface must either copy into caller memory or document a release function with the same care.