Fault Isolation Between Tenants in One Address Space

Running two untrusted programs in separate processes gives isolation by construction: the MMU prevents one from addressing the other’s memory, and the kernel arbitrates. It also costs a context switch for every interaction and a separate set of page tables for every tenant. When the interactions are frequent and the tenants are numerous, the cost dominates, and the alternative is to put both in one address space and enforce the separation in software.

The 1993 argument, still the reference point

Wahbe, Lucco, Anderson, and Graham described software fault isolation for the case where a module’s machine code is rewritten so that every memory access is checked against the module’s own data region. The rewriting inserts a sequence before each store that verifies the target address lies within the region, and the sequence is a few instructions. Their measured cost was on the order of a few percent for the checks, with the sandbox’s benefit being that a fault in one module cannot corrupt another’s data and cannot jump into another’s code.

The mechanism has three parts, and all three are needed.

Address masking or checking. A store’s target is forced into the module’s region before the store executes. Masking replaces the high bits of the address with the region’s identifier, so the store always lands inside the region. Checking compares the address against the region’s bounds and traps if it is outside. Masking is faster and requires the region size to be a power of two; checking is general and costs a branch.

Code separation. Indirect jumps must be constrained to targets within the module’s code. The rewriting checks the branch target against the code region, or uses a table of valid entry points, so that a corrupted function pointer cannot transfer control to another module’s code. Without this, the data isolation is pointless: a module that jumps into another module’s code executes with the other’s data addresses.

A protected scratch area. The rewriting needs registers to perform the checks, and those registers must be saved and restored around the module’s code, or the module could observe and corrupt the check sequence’s state. The register discipline is part of the trusted base.

Why the hardware does not help

The obvious objection is that the MMU already does this, so software isolation is redundant. The objection misses what the hardware cannot express: two tenants in one address space share one page table, so there is no per-tenant page protection to apply. Protection is per-page, and the tenants’ data may be interleaved in the same pages if the allocator so chooses.

Two techniques recover some hardware support.

Reserve a large region per tenant and map it sparsely. If each tenant’s heap is placed in its own multi-gigabyte virtual reservation, then a tenant’s pointer arithmetic, which is 32-bit in many designs, cannot address another tenant’s reservation at all. The reservation need not be backed by physical memory; only the pages actually used are committed. This is the same trick as a pointer-compression cage, and it converts an arbitrary 32-bit offset into an offset that is provably within one reservation.

Use hardware features that do exist, where available. Intel’s Memory Protection Keys allow a page to be assigned to one of a small number of protection domains, and the current domain is held in a register that can be changed without a syscall. The number of domains is small, and the keys protect against data access, not against execution, but they let a runtime switch a whole tenant’s pages to inaccessible while another tenant runs, at the cost of one register write.

Neither is a substitute for the check, because the check is what enforces the boundary within a single instruction stream. The reservations reduce the frequency with which the check can fail, and the keys reduce the window in which a cross-tenant access is possible, but the software check remains the guarantee.

Per-tenant heaps and the allocator’s role

If two tenants allocate from one heap, a defect in one tenant’s allocation path can return a block that belongs to the other. The allocator’s metadata is the most attractive corruption target in the system, because a single overwritten free-list pointer yields an arbitrary write.

The structural answer is a per-tenant allocator with a per-tenant heap. Each tenant’s heap is a separate reservation, and the tenant’s allocations come only from it. The allocator’s metadata lives inside the reservation, so the tenant can corrupt its own metadata and nothing else. Corruption of its own metadata is a fault in that tenant, which the runtime detects when the tenant’s next allocation fails or when a guard page is hit.

The runtime must then ensure that a tenant’s pointers cannot be interpreted by another tenant’s allocator. This is where the type system helps: if a reference is typed by the heap it belongs to, passing it across a boundary is a type error, which is the region discipline described in linearity and aliasing. Where the language cannot express that, the runtime must check at the boundary, and the check is a comparison of the pointer against the target heap’s reservation.

What fault isolation does not give

The phrase suggests more than it delivers, and the gaps are where designs fail.

Confidentiality across tenants. A tenant that can read another tenant’s memory has failed the isolation, but a tenant that can read a shared buffer has not, because the buffer is shared by design. The distinction matters because many runtimes share immutable data, such as compiled code or interned strings, between tenants to save memory. If the shared structure contains anything tenant-specific, the sharing is a channel. A shared string table that deduplicates strings across tenants is a way for one tenant to learn whether another has interned a particular value, by observing allocation behaviour or memory growth.

Availability. A tenant that loops forever, allocates without bound, or holds a lock indefinitely affects the process. Fault isolation addresses memory corruption, not scheduling. Each tenant needs a resource budget enforced by the scheduler, which means the runtime must be able to interrupt the tenant, which in turn means either preemption at safepoints or running the tenant on its own thread.

Timing. Two tenants in one process share caches, branch predictors, and memory bandwidth. A tenant can observe another’s activity through these, as covert channels discusses, and no amount of address-space separation inside the process addresses it.

The runtime’s own state. The runtime’s data structures, the scheduler’s queues, the tenant registry, and the metadata that maps a tenant to its heap are outside every tenant’s reservation, but the code that manipulates them runs in the same address space. A tenant that can influence the runtime’s data through a bug in an intrinsic function has escaped, and the escape is not through a memory access the check covers.

The cost model

The overhead of software isolation has three components, and each can be reasoned about without measurement.

The checks themselves. A store to a checked address is a compare and a conditional branch, or a mask and a store. The compare is predictable, so the branch predictor absorbs it on the hot path; the cost is the instruction slots and the register pressure. In tight loops the register pressure is often the larger cost, because the check needs a register that the loop’s computation also wanted.

The boundary crossings. Every call from a tenant into the runtime, or between tenants, needs a check of the arguments and a switch of the active heap. A crossing is more expensive than an ordinary call by the cost of the argument validation and the heap switch, and the design goal is to make crossings coarse: a batch of operations per crossing rather than one.

The memory reservations. Each tenant’s reservation consumes virtual address space, and the address space is finite: 47 or 48 bits of usable virtual address on current x86-64 implementations, which allows a few tens of thousands of multi-gigabyte reservations. A runtime hosting many tenants must either shrink the per-tenant reservation or multiplex tenants over a smaller set of reservations, and multiplexing reintroduces the need for the check.

The trade against processes is therefore not “software isolation is cheaper” but “software isolation trades per-crossing kernel cost for per-access check cost”. A workload with many small crossings and few memory accesses per crossing favors the in-process design; a workload with large data transfers and few crossings favors separate processes, where the transfer is a single copy and the checks are absent.

Where the design converges

The modern instances of this pattern are WebAssembly engines hosting many modules in one process, and language runtimes hosting plugins in one heap. Both converge on the same structure: a per-instance linear memory or heap in its own reservation, validated or compiled code whose accesses are checked, an import surface that is the only way out, and a scheduler that can terminate an instance.

The convergence is not accidental. The four parts answer four questions: where does the tenant’s data live, how are its accesses constrained, how does it reach the outside, and how is it stopped. A design that answers three of the four has a hole at the fourth, and the hole is usually the one that a real deployment discovers.