The Runtime Cost of Indirection at Module Boundaries

A module boundary is a place where the compiler stops reasoning. It stops because it cannot see across the boundary, because the target is not known until run time, or because the representation on the other side is deliberately opaque. Each of those is a design choice with a reason, and each has a cost that is paid per interaction. The useful question is not whether indirection costs something — it does — but what the cost is proportional to, and whether the interaction rate makes it significant.

The cost model has three terms: a fixed per-crossing overhead, a lost-optimisation term that depends on what the compiler would have done with visibility, and a data-movement term that depends on how much state has to be copied or re-encoded to cross. The first term is small and easily measured. The second is large and invisible in source. The third is usually the dominant one in systems that cross process or network boundaries, and it is frequently underestimated in systems that cross only a function pointer.

What the compiler gives up when it cannot see

Inlining is not primarily about saving the call instruction. A call costs a few cycles on a modern core, and the return is usually predicted correctly. The value of inlining is that the callee’s body becomes visible at the call site, which permits constant propagation into it, elimination of branches whose conditions are now known, removal of dead stores, scalar replacement of aggregates, and vectorisation across what used to be a call boundary.

None of that happens through an opaque interface. A function called through a virtual table, a function pointer, or an exported symbol is compiled independently, against the general case. Its parameters must be materialised in registers or memory because the caller cannot know which ones the callee reads. Its return value must be written to a location the caller can address. The cost is not one call; it is the difference between specialised code and general code, and it can be a factor of several in tight loops even when the arithmetic is identical.

Link-time optimisation recovers some of this within a single binary, because the compiler can see across translation units at link time. It does not recover across a dynamic library boundary unless the library is built with the same toolchain and the build enables cross-module optimisation, and it does not recover across a process or network boundary at all. The boundary’s cost is therefore a function of how far apart the two sides are compiled, which is the same axis the ABI boundary discussion describes at the binary level.

The price list, approximately

The following are order-of-magnitude figures for common x86-64 and ARM64 hardware, not measurements of any particular program. They are useful for deciding which term dominates, not for predicting a speedup.

Operation	Approximate cost
Direct call and return, predicted	a few cycles
Indirect call, correctly predicted	a few cycles plus one load for the target
Branch misprediction	on the order of 15–20 cycles
L1 cache hit	about 4 cycles
L2 cache hit	roughly a dozen cycles
Last-level cache miss to DRAM	hundreds of cycles
Small heap allocation and free	tens of cycles, plus allocator contention under concurrency
System call, no I/O	hundreds to a few thousand cycles
Cross-process round trip	thousands of cycles at minimum, dominated by scheduling
Network round trip, same datacenter	sub-millisecond, four to six orders of magnitude above a local call

Two conclusions follow. First, a single boundary crossing in a cold path is irrelevant; the design should not be distorted to avoid it. Second, the dominant risks are interactions per unit of work and the cache behaviour of the data being passed, not the call instruction. An interface called once per request can afford almost any amount of indirection. An interface called once per element of a large collection cannot, and the fix is usually to change the interface’s granularity rather than to remove the abstraction.

Virtual dispatch and the shapes of call sites

A virtual call loads a function pointer from the object’s vtable and jumps through it. The load is usually a cache hit because the object was just touched; the jump is predictable when the same type appears repeatedly.

Processors predict indirect branches by history. A call site that has seen only one target is effectively direct after the first few executions, and some just-in-time compilers exploit this by emitting a type guard and an inlined body, falling back to the generic path when the guard fails. A call site that alternates between two types is still predictable. A call site that sees many types — a heterogeneous collection of shapes or visitors — becomes a misprediction source, and the cost moves from a few cycles to the misprediction penalty on a meaningful fraction of calls.

This produces a non-obvious design rule: polymorphism is cheap when the set of implementations at a call site is small and stable, and expensive when it is large and varied. The same interface can be an excellent choice in a request handler and a poor one in a loop over a million heterogeneous nodes. Neither the interface nor the language changed; the call-site distribution did.

Statically typed languages with whole-program knowledge can devirtualise when the type is provably single, which is why sealing a class or marking a method final sometimes produces a measurable improvement in a hot path and sometimes produces none. The gain depends on whether the compiler can prove the type at that call site, which it usually can when the object was constructed a few lines earlier and cannot when it arrived as a parameter typed by the interface.

Copies, boxing, and mapping

The data-movement term is where most of the real cost lives, and it is the one that structural diagrams omit.

Passing a large structure by value across a boundary copies it, unless the language’s calling convention passes it by hidden reference for large aggregates and the callee treats it as read-only. Passing it as an interface in a language with value types and interfaces may box it, allocating a heap object and copying the fields into it. Returning a collection may copy the collection, and returning a view may expose internal state that the module wanted to hide. Each of these choices is a boundary decision with a memory-traffic consequence.

The same effect appears at a higher level as mapping. A layered design in which each layer has its own representation of the same entity — a database row, a domain object, a transport DTO, a view model — performs a field-by-field copy at each boundary. The copies are individually cheap and collectively expensive: they allocate, they touch memory that would otherwise stay in cache, and they double the number of objects the garbage collector or allocator must track. The architectural benefit is real isolation, and the cost scales with the number of fields, the number of entities, and the frequency of crossing.

A per-row database query is the extreme form of the same mistake. Each row is a boundary crossing to the database, and the per-crossing cost is a network round trip rather than a function call. The fix is batching: one query for a set of rows, one call for a batch of items. The interface changes from get(id) to get_many(ids), and the number of crossings drops from n to one, which is where the network cost model and the in-process cost model agree.

Granularity as the primary lever

Because per-crossing overhead is roughly constant and the lost-optimisation term is per-call-site, the controllable variable is how many times the boundary is crossed per unit of work. Three patterns reduce it without weakening the boundary.

Batch operations. Replace a loop of calls with one call over a collection. The interface becomes slightly less general and much cheaper to use in bulk, and the implementation can amortise setup, lock acquisition, and allocation across the batch.

/* Per-item writes: one syscall, and one boundary crossing, per element. */
for (size_t i = 0; i < n; i++)
    write(fd, items[i].data, items[i].len);

/* Scatter-gather: the same bytes, one crossing, up to IOV_MAX entries. */
writev(fd, iov, (int)n);

writev does not remove the per-element bookkeeping; it removes the per-element transition into the kernel, which is the part the boundary charges for.

Buffer handoff. Pass a buffer that the callee fills, rather than returning one item at a time. This is the standard shape of system-call interfaces — read into a caller-provided buffer — and it exists precisely because the crossing is expensive and the number of items is not known in advance.

Coarse-grained lifecycle. Keep an object alive across a sequence of operations instead of creating and destroying it per operation. A prepared statement, an open cursor, or a compiled query holds state on the far side of the boundary so that repeated work does not repeat the setup.

Each of these trades interface purity for a lower crossing count. The trade is worth making when the boundary is expensive relative to the work: a network, a process, a syscall. It is usually not worth making between two classes in the same binary, where the crossing costs a handful of cycles and the batch API costs readability at every call site.

Where the cost model misleads

The model treats crossings as independent, and they are not. A boundary that forces an allocation per call also increases pressure on the allocator and the cache, and the resulting slowdown may appear in code that never crosses the boundary. A virtual call that is cheap in isolation may be expensive because it prevents the compiler from keeping a value in a register across the call, spilling it to the stack instead. Reasoning about these interactions from a table of latencies is unreliable; the table identifies candidates, and a profiler with call counts identifies the ones that matter.

The model also ignores the cost of the alternative. Removing a boundary means duplicating the logic it protected or exposing the representation it hid, and the maintenance cost of that duplication appears later as inconsistency. A performance argument for merging two modules has to account for the change cost that information hiding was buying, not just the cycles it was charging.

Finally, the numbers are hardware-specific and move with each generation. Cache latencies, branch predictor capacity, and allocator behaviour differ across cores and across workloads. The stable part of the model is the shape — fixed overhead times frequency, plus lost optimisation, plus data movement — and the shape is enough to decide whether an interface should be coarse or fine. The constants are for the profiler, not for the design review.

The boundary is a contract about variation

An indirection is worth its cost when something on the other side varies: an implementation, a deployment target, a test double, a version. When nothing varies, the indirection is a tax on every call for a flexibility nobody uses. The same test applies at every level — virtual function, shared library, service — and it is the reason the cheapest design is usually the one whose boundaries each correspond to a decision that actually changes.

That criterion returns the discussion to where it started. Coupling and cohesion describe the structure, interface contracts describe what crosses it, and the cost model here describes what the crossing charges. A design that gets all three right is one where the expensive boundaries are the ones that hide the most volatile decisions, and the cheap ones are where nothing needed hiding at all.