Tenant Isolation and the Channels Performance Interference Travels Through

Isolation in the security sense is a solved problem in every design that gives each tenant its own address space and its own interrupt attribution: a tenant cannot read another tenant’s memory, cannot forge another tenant’s interrupts, and cannot issue work attributed to another tenant. Isolation in the performance sense is not solved, and no design solves it completely. The reason is structural: the resources that determine throughput are, in most designs, still shared even when the address spaces are not.

Enumerating those resources and the mechanism by which each is shared is the only way to reason about interference, because the controls that exist are attached to specific resources and are useless against interference that arrives through a different one.

The four shared resources and how each leaks

The first is device memory bandwidth. Every tenant’s memory traffic converges on the same memory controllers unless the design partitions them, as fixed-slice partitioning does. Where they are shared, a tenant that issues a large number of memory requests occupies the controller’s queues and the row buffers of the banks it touches, and a co-tenant’s requests queue behind them. The interference is not proportional to the aggressor’s useful bandwidth: a tenant issuing scattered accesses that miss in row buffers consumes more controller occupancy per byte delivered than a tenant issuing sequential ones, so the same delivered bandwidth can cost a co-tenant very different amounts of delay.

The second is last-level cache capacity. A cache is shared by address, not by tenant, so an aggressor streaming a large working set evicts a co-tenant’s lines. The victim’s miss rate rises even though its own access pattern has not changed, and the effect is largest for victims whose working set is just small enough to have fit before. This is the same mechanism that makes co-running processes on a CPU interfere, and the mitigations are the same shape: partition the cache by address range or by tag, or accept the interference and account for it.

The third is the interconnect between host and device. A tenant transferring data over PCIe consumes the link’s bandwidth in both directions, and a link is a shared serial resource with a fixed number of lanes. Two tenants each transferring at half the link rate is fine; two tenants each attempting the full rate produce a queue in which both are delayed. Because the link is also the path for doorbell traps and interrupt delivery in mediated designs, saturation of the link degrades control-path latency as well as data throughput.

The fourth is power and clock, which no partitioning scheme separates. A GPU has a power budget and a thermal envelope, and the clock the device runs at is a function of how much power it is drawing. A tenant that runs a dense compute kernel drives the device to its power limit, and the device responds by reducing clock, which slows every tenant on the card including ones that were not consuming power. This is interference with no address-space component at all: the victim’s kernels simply execute more slowly because the clock is lower.

Why accounting by time is not accounting by work

The scheduler in a mediated or time-sliced design usually accounts in units of time: a tenant that holds the device for a millisecond is charged a millisecond. That accounting is correct for fairness in the sense of opportunity and wrong for fairness in the sense of outcome, because a millisecond of device time is worth different amounts of work depending on what the co-tenant is doing.

If tenant A’s kernel saturates memory bandwidth and tenant B’s kernel is compute-bound, then B’s throughput during its own slot is barely affected by A, while A’s throughput during its slot depends on whether B’s kernel left the caches full of A’s data. Charging both the same time produces a distribution of completed work that neither tenant would call fair. Correcting it requires the scheduler to know the resource each tenant consumes, which means either instrumenting the device’s performance counters per context or accepting an approximation.

Per-context counters are the reliable route and they are not universally available. A device that can attribute memory transactions, cache misses, and execution cycles to a context can be scheduled on measured consumption; one that can only attribute time cannot. This is a hardware capability that differs by generation and by vendor, and its absence is what makes static partitioning the fallback, since with no shared scheduling the question of fairness does not arise.

The controls that exist

Rate limiting at the scheduler is the blunt control. A tenant is admitted for at most a specified fraction of time, or its submissions are delayed to enforce a target rate. This bounds the tenant’s own throughput and, indirectly, its interference, but the relationship between time share and interference is not linear and not stable.

Cache partitioning by address window is the sharper control where the hardware supports it. If the L2 is divided into slices and each slice serves a contiguous physical address range, then a tenant’s pages can be placed in a subset of slices and its cache footprint bounded. The cost is that the tenant’s effective cache is smaller than the device’s, so a tenant that would have benefited from the full cache is slowed by the partition even when it is alone. This is a capacity-for-predictability trade, and it should be made deliberately rather than by default.

Bandwidth throttling at the memory controller is possible on some devices, usually by limiting the number of outstanding requests a context may have. It directly targets the resource that causes most interference, and it has the property that a throttled tenant’s own performance degrades gracefully: memory-bound kernels slow down in proportion to the throttle, compute-bound kernels barely notice. The difficulty is setting the limit, because the right value depends on the co-tenant mix and changes as tenants come and go.

Clock and power controls are the last resort. Capping the device’s clock bounds the total power draw and therefore bounds the thermal interference, at the cost of slowing everyone. It is the correct response when the alternative is a device that throttles unpredictably, and it is not a substitute for partitioning.

Detecting interference after the fact

The practical problem is that a tenant’s operator sees a slowdown and cannot tell which resource caused it. Three measurements separate the common cases.

Compare the tenant’s performance when running alone against its performance with the co-tenant present. The ratio is the total interference, and it is the only figure that matters to the operator.

Then compare the tenant’s achieved memory throughput against its achieved instruction throughput. If memory throughput is unchanged and instruction throughput fell, the interference is in execution resources or clock. If memory throughput fell and the access pattern is unchanged, the interference is in the memory system.

Then compare the miss rate against the working set size. A victim of cache eviction shows a miss rate that rises sharply as the co-tenant’s working set grows past the cache capacity, while a victim of bandwidth contention shows a miss rate that is roughly constant and a latency that rises. These two have different remedies: partitioning the cache helps the first and does nothing for the second, while throttling the aggressor’s requests helps the second and leaves the first intact.

The measurement itself must be done with counters the device exposes per context, because aggregate counters conflate the tenant’s own behaviour with its neighbours’. Where per-context counters are absent, the only sound approach is controlled experiments: run the tenant with and without a synthetic co-tenant of known intensity and fit the response. That is a laboratory procedure, not a runtime control, which is why a design that has to guarantee a tenant’s throughput needs hardware partitioning rather than measurement and delay.

The boundary that cannot be enforced

A tenant that is denied bandwidth can still consume it indirectly by holding the device and issuing requests, and a tenant that is denied cache can still evict by touching memory. None of these designs prevents a tenant from consuming shared capacity up to the limit of its own allotment. What the designs differ in is whether the allotment is a hard boundary in hardware, as in SR-IOV slices and fixed partitions, or a policy enforced by a scheduler that can only observe and delay.

The honest formulation is that partitioning buys isolation of capacity and sharing buys utilisation, and the two cannot both be maximal. A card divided into fixed slices guarantees each tenant its share and wastes whatever a tenant does not use. A card shared by a scheduler uses everything and guarantees nothing beyond the schedule. The two policies are alternatives rather than layers: attempting both on the same device produces the guarantees of neither.