Steal Time and the Mechanics of CPU Oversubscription

A host with sixty-four physical cores can sell far more than sixty-four vCPUs. The ratio between the two numbers is the oversubscription factor, and it is the central economic mechanism of the instance market: it is what allows a provider to price a small instance below the amortized cost of the core it nominally contains. The cost of that ratio is paid by tenants in the form of descheduled time, and the mechanism by which they observe it is a counter called steal time.

Run queues and the right to run

Each vCPU is a host thread. When it is runnable — meaning the guest has work for it and the guest’s own scheduler has not idled it — it sits on a host run queue. The host scheduler picks among runnable threads according to its policy, which in the general-purpose case is a weighted fair-share scheduler: every thread has a weight derived from a configured share, and the scheduler tracks accumulated virtual runtime so that threads with equal weight receive equal service over time. A thread that has run less than its peers is picked first.

Oversubscription changes only the number of threads competing, not the policy. With a factor of one, a runnable vCPU thread almost always finds a core immediately. With a factor of four, three of every four runnable vCPU threads are waiting at any moment when all tenants are busy. The waiting is not a bug in the scheduler; it is the scheduler doing exactly what it was configured to do, distributing a fixed quantity of core time among more claimants than there are cores.

The critical subtlety is that the guest’s scheduler does not know this. Inside the guest, four vCPUs appear to be four independent processors. The guest kernel dispatches runnable tasks onto them assuming each will make progress at full speed. When the host deschedules a vCPU thread, the guest’s vCPU simply stops executing, mid-instruction-stream, with no fault and no notification. The guest kernel’s own run-queue accounting continues to believe that the task it dispatched is running. It is not.

How the guest finds out

The guest learns about descheduling through a paravirtualized accounting interface. The hypervisor maintains, per vCPU, a count of the time during which the vCPU was runnable but not running, and it exposes that count to the guest through a shared memory structure that the guest kernel reads without causing an exit. On x86 the structure is a per-CPU page whose fields are updated by the hypervisor and read by the guest; the guest kernel samples the field at each timer tick and adds the delta to a per-CPU steal counter.

Because the interface is a shared page rather than a trap, sampling is cheap, but sampling has a resolution limit. The guest sees steal time quantized to its tick, which is typically in the millisecond range or finer depending on the kernel’s configured frequency. A vCPU that was descheduled for 200 microseconds between two ticks contributes to the same bucket as one descheduled for 2 milliseconds, and neither the guest nor its monitoring can distinguish them after the fact. This is why steal time is useful as a trend indicator and unreliable as a per-request latency explanation.

The accounting also distinguishes states that the guest cannot otherwise see. A vCPU can be runnable in the guest but not running because the host scheduled another thread; that is steal. A vCPU can be idle in the guest and simply not scheduled because there is nothing to run; that is not steal, and it is invisible to the guest as anything but idle time. A vCPU can be blocked in the host on I/O or on a hypercall; that is neither. Collapsing all three into one number is the most common mistake in reading the counter.

# Per-vCPU state as the guest sees it, sampled at each tick.
#
#   guest_idle      -> vCPU halted; host may not schedule it at all
#   guest_runnable  -> guest has a task for this vCPU
#     host_running  -> it is executing on a core           (user/system time)
#     host_waiting  -> another thread holds the core        (steal time)
#   guest_blocked   -> vCPU waiting on I/O or a hypercall
#
# Steal time counts only the second-level split under guest_runnable.

The reason the distinction is enforceable at all is that the hypervisor knows why the thread is not on a core. It knows whether the guest executed a halt instruction, whether the thread is blocked in the host kernel, and whether it was preempted while runnable. The guest, by contrast, only knows what its own scheduler did.

What a high steal number means and what it does not

A sustained steal fraction above a few percent indicates that the host is contending for cores. The usual causes are the tenant’s own oversubscription — running more busy threads than it has vCPUs, which is harmless in the sense that the guest’s scheduler will time-slice them and the host sees at most one runnable thread per vCPU — and other tenants on the same host, which the guest cannot control.

The distinction between those two is visible in the guest’s own load average. If the guest has four vCPUs and its load average is four, all four vCPU threads are runnable and any steal is host contention. If the load average is sixteen, the guest’s scheduler is already time-slicing, and the host sees only four runnable threads, so the guest is slowing itself down before the host does anything. The first case is a sizing problem; the second is an application concurrency problem that looks like a sizing problem.

A high steal number does not, by itself, prove that the provider is at fault, and a low one does not prove that performance is unaffected. The counter measures time not spent running, not the quality of the time that was spent. Two effects are invisible to it. First, a vCPU that runs on a core shared with another tenant’s vCPU through simultaneous multithreading contends for execution units without ever being descheduled; the counter stays near zero while throughput per cycle falls. Second, a vCPU that is migrated between physical cores loses its warm caches and its translation cache entries; it was never descheduled, but it pays a cold-start penalty on each migration. Neither appears in steal time. The first effect is examined in microarchitectural contention between tenants, which covers the shared resources a time-based quota cannot bound.

The scheduling latency that matters more than the average

For latency-sensitive work the relevant quantity is not the mean steal fraction but the distribution of descheduling intervals. A vCPU descheduled for one percent of the time in a thousand one-millisecond slices behaves very differently from one descheduled for one percent in a single ten-millisecond slice. The first adds a millisecond of jitter to one request in a hundred; the second stalls an entire batch of requests for ten milliseconds.

The host scheduler’s placement decisions determine which pattern a tenant gets. A fair-share scheduler that gives each runnable thread a bounded slice and rotates produces many short deschedules. A scheduler that lets a thread run until it blocks or exhausts a longer slice produces fewer, longer ones. Providers tune this globally and do not expose it, which is why two instances of the same type can show the same steal percentage and very different tail latency.

There is a mitigation available to the guest, and it is the same one used on bare metal: reduce the number of runnable threads per vCPU so that each vCPU thread spends most of its time as the only claimant. A busy-polling thread that never yields defeats this, because it keeps its vCPU runnable continuously and forces the host to time-slice it against every other runnable thread on the machine. Setting a polling budget, or letting the thread block on the completion queue when it has no work, converts a permanently runnable vCPU into one that is frequently idle and therefore not competing.

Why the ratio is not advertised

Providers do not publish the oversubscription factor of a given instance family, for two reasons. The factor is not a constant: it depends on the live population of the host, which changes as instances are created and destroyed. And the factor is not uniform across instance families. Families marketed for sustained compute tend to run at a lower ratio than families marketed for bursty or low-cost workloads, because the former are expected to keep every vCPU busy while the latter are expected to idle most of them.

The observable consequence is that the same nominal vCPU count can deliver different throughput depending on which family and which host it lands on. Benchmarking an instance type therefore measures the host it happened to run on at that moment as much as the type itself. This is why providers that offer dedicated or isolated tenancy do so as a separate product: the guarantee being sold is not more cores but the absence of other runnable threads on them, which is a property of placement rather than of the hardware. The placement machinery that decides which host an instance lands on is described in placement, packing, and fragmentation, and the accounting interface itself is documented in the kernel’s scheduler documentation rather than in any provider API.