What CPU Quotas Cannot Bound: Cache, Bandwidth, and SMT Contention Between Tenants

The accounting model of a shared host is a time-slicing model: each vCPU receives a share of core time, and the share is enforced by the host scheduler. The model is exact for time and silent about everything else. Two vCPUs that each receive fifty percent of a core’s cycles can still have wildly different throughput if one of them shares the core’s execution units, the last-level cache, the memory bus, or the translation lookaside buffer with a neighbour that is hammering them.

The time-slice model and its blind spot

A fair-share scheduler equalizes the time each runnable thread spends on a core. It does not equalize the work that time produces, because work per cycle is a function of the microarchitectural state the thread finds when it runs.

Three mechanisms break the equality. The first is simultaneous multithreading. A physical core exposes two (or more) hardware thread contexts, and when both are occupied they share the core’s execution units. Two threads issuing from the same execution ports at the same time contend; a thread running alone uses the core’s full width. The scheduler’s time accounting sees two vCPUs each receiving a full core’s worth of time, while the hardware has delivered something closer to one and a quarter cores’ worth of throughput in total.

The second is the cache hierarchy. The last-level cache is shared among all cores on a socket, and it is not partitioned. A tenant whose working set exceeds its private caches evicts its neighbours’ lines, and a neighbour whose working set would otherwise fit now misses. The scheduler’s accounting records the eviction victim as having received its full time slice; it did, and its instructions stalled on memory.

The third is memory bandwidth. The memory controller serves the whole socket, and a tenant streaming through a large array consumes a disproportionate share of it. Other tenants’ latency-sensitive accesses queue behind the stream. This is the mechanism behind the observation that a latency-critical service can be degraded by a batch job running on a different core of the same machine, with no shared cache lines and no scheduler interaction at all.

Measuring the leak

None of these effects appear in steal time, because the vCPU was running. They appear instead as a drop in instructions retired per cycle, or as a rise in memory stall cycles, and both are visible in the guest’s own performance counters if the provider exposes them.

Hardware performance counters are per-core and can be configured to count only the events of the owning context, which makes them usable for attribution within a guest. A guest measuring its own cycles and instructions sees a ratio that falls when a neighbour is active, and it can correlate that fall with the neighbour’s activity only indirectly, since it cannot see the neighbour at all. The correlation is usually visible as a step change at a time when the guest’s own workload did not change.

The provider’s side has better visibility. A host-side monitor can read the counters for both contexts on a shared core and compute how much of each tenant’s stall time is attributable to contention. That measurement is the basis for the placement decisions described below, and it is rarely exposed to tenants in a form that would let them attribute their own slowdown.

# What a tenant can observe, and what each signal does and does not prove.
#
# IPC drop, own workload unchanged     -> contention or frequency change
# LLC miss rate rise                   -> working set grew, or lines evicted
# memory stall cycles up, IPC down     -> bandwidth queueing from a co-tenant
# steal time zero, throughput down     -> the scheduler is not the cause

The last line is the diagnostic that matters. A tenant seeing throughput loss with zero steal time has ruled out descheduling and should look at the microarchitectural signals, not at the provider’s scheduling policy.

Placement decisions that bound the leak

The primary defence is not enforcement but placement. If the provider knows which instance types are latency-sensitive and which are throughput-oriented, it can avoid co-locating them on the same physical core or socket.

At the core level, the decision is whether to place a tenant’s vCPU on an SMT sibling that is already occupied. A provider that leaves one hardware thread per core idle for latency-sensitive instances gives up roughly a third of the core’s throughput in exchange for removing the execution-unit contention. A provider that packs both siblings sells the same physical core twice and accepts the throughput loss for both tenants.

At the socket level, the decision is which tenants share a last-level cache and a memory controller. A batch tenant with a large streaming working set and a latency-critical tenant with a small hot set are the worst pairing, because the streaming tenant evicts the hot set and saturates the bandwidth. The best pairing puts tenants with similar access patterns together, or isolates the streaming tenant on a socket of its own.

At the memory level, the decision is which pages land in which cache sets, which is not controllable without cache-allocation technology. Some processors expose a way to partition the last-level cache into classes and assign cache ways to each class, which converts a shared resource into a set of reserved regions. The mechanism is available to the host and invisible to the guest unless the guest is given a way to request a class. Where it is unavailable, the only lever is placement.

Why quotas do not substitute

It is tempting to think that a stricter CPU quota would contain a noisy neighbour. It does not, because the quota bounds the neighbour’s time and the contention is a function of what the neighbour does with that time. A neighbour that runs a streaming kernel for ten percent of the time can still evict a co-tenant’s entire cache during those ten percent, and the co-tenant’s subsequent misses are paid over the remaining ninety percent.

The reverse is also true: a neighbour that is well behaved but has a large working set will degrade a co-tenant that has a small one, simply by existing. The victim’s own quota has not changed, its own time has not changed, and its performance has fallen by a factor that depends on the ratio of the two working sets and the associativity of the shared cache.

The consequence is that capacity planning for shared hosts cannot be done in vCPU-equivalents alone. Two instances with identical shapes on the same host can have different effective capacity because of who else is there, and the difference is not visible in any per-instance metric the provider publishes. This is the argument for dedicated tenancy as a product: the thing being sold is a placement guarantee, and the guarantee is about the absence of co-tenants rather than about the hardware. The placement machinery is where that guarantee is either honoured or quietly broken, since a node that is co-located by the packing heuristic defeats the isolation that was purchased.

Failure modes that look like application bugs

Three symptoms are commonly misattributed.

A latency-critical service with a p99 that degrades at unpredictable intervals, with no change in its own traffic and no steal time, is the classic co-tenant contention signature. The intervals correspond to another tenant’s periodic job, and the service cannot see it. The fix is either to move the service to isolated capacity or to reduce its sensitivity by shrinking its working set.

A throughput-oriented batch job that runs slower than its instruction count predicts, with high last-level cache miss rates on a working set that should fit, is the signature of eviction by a neighbour with a larger set. Reducing the batch job’s footprint or changing its access pattern to improve locality helps; so does running it on a host with fewer co-tenants.

A service that performs well in isolation and poorly in a shared cluster, with no saturation in its own metrics, is often suffering from memory bandwidth contention rather than from any of the above. The tell is that the degradation scales with the co-tenant’s memory intensity rather than with its CPU usage, and that it is worst for workloads whose instructions-per-cycle are dominated by memory-level parallelism.

None of these are fixed by adjusting the request and limit pair, because that pair governs the scheduler’s time accounting. They are fixed by changing placement, changing the workload’s footprint, or buying isolation. The one exception is the case where the co-tenant is inside the same instance, in which case the contention is between the tenant’s own containers and the remedy is the same analysis applied one level down.

The direction the hardware is moving

The trend is toward more sharing, not less. Core counts per socket rise, which increases the number of tenants per last-level cache. Memory bandwidth per core falls as cores are added, which makes the shared resource scarcer relative to demand. SMT is present on nearly every server part, and the provider’s choice is only whether to sell both siblings.

The counter-trend is that hardware is gaining mechanisms for partition rather than fair sharing: cache way allocation, memory bandwidth monitoring with per-class throttling, and address-space identifiers that reduce translation-cache interference. These mechanisms move the problem from placement to configuration, which is an improvement in principle and a new source of complexity in practice, since a misconfigured partition is worse than no partition at all.

What does not change is the accounting mismatch. A scheduler can only enforce what it can count, and the resources that leak between tenants on a shared core are counted in cycles and misses rather than in time. Until enforcement operates on those units, isolation on shared hardware is a placement property, and the compute model’s finest-grained unit is the physical host rather than the vCPU.