Designing a C ABI That Survives Version Changes

Binary compatibility is a constraint on change, not a property of code. A library is compatible with a caller when the caller’s existing machine code continues to work against the new library, and that is a much stronger requirement than source compatibility. The caller cannot be recompiled, cannot be asked to change, and in the case of a language binding may not even be inspectable.

The constraints that follow are narrow and specific. They concern the size and layout of every type that crosses the boundary, the name and signature of every exported symbol, the ownership of every value transferred, and the threading and failure behaviour of every entry point. Each of those can be changed compatibly, but only in particular ways, and the ways are different for each.

What a caller’s machine code has baked in

A compiled call site contains constants the library cannot revise. It contains the symbol name, resolved at link time. It contains the number and class of arguments, which determine which registers are written. It contains the size of any struct passed by value, because the caller reserves that much space in its own frame, and that size follows from the register classification rules as much as from the layout. It contains the offset of every field it reads, because field access compiles to a fixed displacement. And it contains the size of any struct the caller allocates, because the allocation happens on the caller’s side.

The first three are visible in the symbol table and the call instruction. The last two are invisible: a struct layout is not recorded anywhere the dynamic linker checks. This is why adding a field to a struct is a compatible change only when the struct is opaque, and why adding a parameter to a function is never compatible, even when the new parameter has a default in the source.

There is one exception worth naming. A function whose caller passes arguments in registers leaves the unused argument registers holding whatever the caller last put there. If a new version of the function reads an argument that the old caller never set, it reads garbage rather than a default or zero. Some platforms zero the unused registers at the call site, and some do not; relying on either is relying on an implementation detail. The safe form is a new symbol.

Growing a struct without breaking callers

The classic technique is to put the struct’s size in its first field, and to require the caller to initialize it before passing the struct in. The library reads the size and treats any field beyond it as absent, using its own defaults. An old caller sets the size to the old value and the new library ignores the new fields; a new caller sets the larger size and the old library, which knows only the old layout, reads the size and refuses, or reads only the prefix.

This works because the size field is at offset zero and never moves. It costs one field, one initialization requirement, and a check in every entry point that accepts the struct. The requirement that the caller initialize the size is a source of bugs: a caller that allocates the struct with zeroed memory and forgets to set the size presents a struct of size zero, and the library must decide whether to treat that as an error or as a request for defaults.

The alternative is a version number in place of the size, which is smaller but less precise: it tells the library which layout to assume, but not how much memory the caller actually allocated, so a caller that passes a version-3 struct in a version-2-sized buffer cannot be detected. Size-based extension is more reliable for that reason, and it is what most extensible C interfaces use.

A third technique is the reserved-field approach: allocate a fixed number of pointer-sized fields at the end of the struct, documented as reserved and required to be zero. A future version may define one of them. This costs memory in every instance and constrains the library to additions that fit in a pointer, but it requires no initialization protocol and no size check. It is the technique to use when the struct is created by the library rather than by the caller, since the library can then define the fields without any caller involvement.

Symbol versioning and the parallel-entry-point pattern

When a function’s behaviour must change incompatibly, the options are a new symbol name or a versioned symbol.

A new name is the simplest and most portable. db_open becomes db_open_v2, the old symbol remains, and new callers use the new name. The cost is that the old symbol must be maintained, which means the library carries both implementations, and the old one accumulates the bugs that were the reason for the change.

ELF symbol versioning lets one name carry multiple definitions, selected by the version recorded in the reference. A library defines db_open@VERS_1.0 and db_open@VERS_2.0, with the default version applying to newly linked callers. A binary linked against version 1.0 continues to bind to 1.0. The mechanism is implemented through a version definition section and a version need section in the object, and the dynamic linker resolves the reference using both. The advantage is that source code keeps one name; the disadvantage is that the mechanism is ELF-specific, with no equivalent on Mach-O or PE, and that it complicates static linking and symbol interposition.

A related pattern is the dispatch shim: keep the exported name stable and have it call an internal implementation chosen at load time, based on a version query or a capability check. This is what allows one binary to serve callers that expect different behaviours, and it is common in libraries that must run on several operating system versions.

The rules that hold regardless of mechanism

Five rules recur across every stable C interface, and each corresponds to a class of breakage.

Never change the width or signedness of an existing parameter or return type. A function that took int and now takes int64_t changes the register usage and the interpretation of the upper bits. Even widening a return value breaks callers that read only the low half and assume the high half is unspecified.

Never change the ownership of a returned value. If a function returned a borrowed pointer and now returns memory the caller must free, every existing caller leaks; if it returned owned memory and now returns a borrowed pointer, every existing caller frees memory it does not own. Both are silent until the heap is corrupted.

Never change the threading contract. A function that was safe to call from any thread and is now required to be called from one specific thread will be called from the wrong thread by existing code, and the failure appears as a data race rather than as an error.

Never change the failure channel. A function that returned an error code and now throws, or now aborts, or now returns null where it previously returned a valid sentinel, breaks callers that handled only the old channel. The rule is one failure channel per entry point, documented and permanent.

Never remove an exported symbol while any binary that references it may still be loaded. This includes symbols that are no longer documented and symbols that exist only for compatibility. The linker cannot tell an obsolete reference from a current one.

The five rules are conservative in the same direction: they treat the compiled caller as immutable. That is the correct model for a C ABI, because the caller may be a binary built years ago by a toolchain the library’s authors never see.

Encoding the contract where it can be checked

The rules above are conventions, and conventions erode. The mitigation is to put as much of the contract as possible into artifacts that can be compared mechanically.

A header that declares every exported function and every opaque type is the primary artifact: it is the source both sides compile against, and a change to it is visible in a diff. A version script or an export list is the second: it names exactly which symbols are public, so a symbol that accidentally becomes exported because it lost its hidden visibility is caught by comparing the list against the object’s dynamic symbol table. A test that links a small program against the previous version’s headers and runs it against the new library is the third, and it is the only one that exercises the actual binary compatibility rather than the source-level appearance of it.

Generated bindings add a fourth check. If the binding is generated from the header, then the binding’s view of every struct size and field offset is derived mechanically, and a header change that alters a layout shows up as a change in the generated output. A binding maintained by hand has no such signal, which is why hand-written bindings drift.

typedef struct db db;   /* opaque, defined inside the library */

/* Extensible options struct: size first, reserved tail. */
typedef struct {
    uint32_t size;              /* caller sets to sizeof(db_opts) */
    uint32_t flags;
    const char *cache_dir;      /* may be NULL */
    void *reserved[4];          /* must be zero-filled by the caller */
} db_opts;

db *db_open_ex(const char *path, const db_opts *opts);

The struct is passed by pointer, not by value, which removes it from the register-classification rules entirely and lets it grow without changing the call site. size lets the library detect an older caller. The reserved tail lets a future version define fields without a size change at all, as long as the caller zeroed them. Every one of those decisions is a small cost paid at design time to avoid a class of breakage later.

Where stability ends

An ABI can be stable and still be the wrong interface. The constraints above make some designs impossible: an interface cannot change a callback’s signature, cannot widen an identifier, cannot switch from a caller-allocated buffer to a library-allocated one. A library that needs any of those must add a parallel entry point and keep the old one, which is how C libraries accumulate the long tail of functions whose names carry version suffixes.

The other limit is platform scope. An ABI is stable relative to a platform’s conventions, and a library that supports several platforms has several ABIs. A struct’s layout is the same across them only if the field types have the same size and alignment, which is not true for long between 32-bit and 64-bit targets. Fixed-width types in the header are the only way to keep one layout across all of them, and the header must use them even where the platform’s native type would be more idiomatic.

The final limit is the one the earlier articles in this cluster keep returning to: the ABI covers the mechanics of the call and nothing else. It does not say who frees, which thread may call, whether the function may block, or what an invalid handle means. Those are protocol, and protocol is preserved by documentation and by tests rather than by the linker. The interfaces that last are the ones where the protocol is small enough to state completely and mechanical enough to check, which is the same point the calling convention article reaches about the convention itself, one layer down.