Resource Requests, Limits, and What the Scheduler Actually Counts

Every workload declaration carries two numbers per resource, and they answer different questions. The request answers “how much must be free for this to be placed here”, and it is consumed only by the scheduler. The limit answers “how much may this use before something intervenes”, and it is consumed only by the kernel. Nothing enforces the request at runtime, and nothing schedules against the limit. Once that separation is clear, most of the confusing behaviour of a cluster: a node reported as full while its CPU sits idle, a workload throttled although the node has spare capacity, a pod evicted for memory it never requested, follows directly from it.

Two numbers, two enforcement points

A request of 500 millicores and a limit of 1000 millicores means the scheduler will only place the workload on a node with at least 500 millicores unaccounted for, and the kernel will let the workload run for at most one core’s worth of time per period before stopping it until the next period. If the workload uses 200 millicores, the node’s accounting still shows 500 consumed. If it tries to use 1500, the kernel caps it at 1000 and the excess demand shows up as throttling, not as a scheduling failure.

Memory behaves the same way with a different consequence. A memory request is subtracted from the node’s allocatable memory at scheduling time and never reserved in hardware. A memory limit is a hard ceiling enforced by the cgroup’s memory controller: when the group’s usage reaches the limit, the kernel reclaims from the group’s page cache first and, if that is not enough, invokes the group OOM killer rather than letting the allocation succeed. Memory is not time-sliced, so there is no equivalent of throttling: a memory-hungry workload either gets its pages or dies.

CPU limits are enforced as bandwidth. The controller gives the group a quota and a period; when the group’s threads exhaust the quota within the period, they are throttled until the period rolls over. The default period is 100 milliseconds, so a workload with a 100 millicore limit gets 10 milliseconds of CPU time per 100-millisecond window. The granularity matters: a workload that needs 30 milliseconds of CPU in a burst cannot get it within one period even though its average is 300 millicores, because the quota is spent at the 10-millisecond mark. This is the origin of the well-known pattern where a latency-sensitive service with a low CPU limit reports poor tail latency despite low average utilisation: the tail is quota exhaustion inside a single period, not contention with other workloads.

There is no equivalent accounting for a request at runtime. The kernel’s proportional-share weight for a cgroup is derived from the request, so a workload with a larger request gets a larger share of contended CPU, but that is a weight, not a reservation: if the node is idle, a workload with a small request can use the whole node. Only the limit stops it.

Millicores, bytes, and the units that are not units

CPU is expressed in cores, with the milli prefix for fractions; 1000m equals one core, and the scheduler’s internal representation is an integer count of millicores, so fractional cores below one millicore are rounded up at admission. Memory is expressed in bytes, with binary suffixes (Ki, Mi, Gi) and decimal ones (K, M, G) both accepted and distinguished: 1Ki is 1024, 1K is 1000. The distinction that has caused more than one capacity plan to be off by a few percent.

Hugepages are the exception that proves the rule. A hugepage request is a request for a specific page size, and the scheduler accounts it against the node’s preallocated pool for that size, not against general memory. Because the pages are preallocated and cannot be subdivided, a node with free memory can still be unable to satisfy a hugepage request, and the workload stays pending. Extended resources, such as a GPU advertised by a node plugin, are integers only, are never overcommitted, and must appear as both request and limit; the scheduler treats them as opaque counts with no relationship to any real capacity it can measure.

Why only requests are summed

The scheduler’s node accounting is a subtraction. Each node advertises allocatable capacity, which is its total capacity minus what the kubelet reserves for itself and for the system, and the scheduler subtracts the sum of the requests of every workload already bound there. A workload fits if the remainder covers its requests. Limits never enter this calculation.

That choice is what makes overcommit possible and, in most clusters, desirable. If the scheduler summed limits, a cluster could not run workloads whose limits exceed their requests, and almost every workload in a real cluster is shaped that way: a web server requests 200 millicores and limits at 1000, because it needs 200 to be schedulable and might briefly need a full core. Summing requests means the cluster is planned against a figure that reflects steady-state need, and the burst capacity is shared statistically.

The consequence is that a node can be simultaneously “full” from the scheduler’s point of view and mostly idle in reality. A workload that requests a core and uses a tenth of it occupies a core of scheduling capacity for its lifetime. This is not a bug in the accounting; it is the accounting choosing predictability over utilisation. The scheduler cannot know future usage, so it plans against the declared figure, and the declared figure is the operator’s statement of what the workload needs in the worst realistic case. Clusters with badly calibrated requests are either wasting most of their capacity or overcommitting it, and the scheduler has no way to tell which: the number it trusts is the number it was given.

The reverse error is equally common. A workload whose request is far below its real usage is schedulable everywhere and then competes for CPU on a node the scheduler believed was comfortable. Because the request also sets the cgroup’s share weight, such a workload is not merely crowded; it is crowded and low-weight, so it loses the contention it creates.

Three classes, one eviction order

The relationship between requests and limits for all containers in a pod determines its quality-of-service class, and the class determines what happens when the node runs short.

A pod is Guaranteed when every container sets both CPU and memory requests and limits, and the two are equal for each resource. It is Burstable when at least one container sets a request or limit but the pod does not meet the Guaranteed condition. It is BestEffort when no container sets any request or limit for CPU or memory. The classification is computed by the API server at admission and stored in the pod’s status, and it is not advisory: the kubelet uses it to order eviction.

Under node memory pressure, the kubelet ranks pods for eviction by consumption relative to request, with BestEffort pods first, then Burstable pods that exceed their requests, then Burstable pods within their requests, and Guaranteed pods last. The ranking within a class uses the pod’s memory usage against its request, so a Burstable pod using twice its request is a more attractive eviction target than one using exactly its request. Guaranteed pods are not immune; they are simply last, and a node that is out of memory with only Guaranteed pods on it will still evict.

The class also affects CPU contention through the derived share weights, and it affects the OOM score adjustment the container runtime applies, which biases the kernel’s own OOM killer toward lower classes when a cgroup-level kill is not what happens. Both mechanisms point the same way: the pod that declared the least is the pod that pays first.

Init containers, sidecars, and the arithmetic of a pod

A pod’s effective request is not the sum of its containers’ requests in every case, and the exceptions are worth knowing because they change the number the scheduler sees.

For a pod with init containers, the effective request for a resource is the maximum of two quantities: the sum over all regular containers, and the maximum over init containers of (that init container’s request plus the sum of the requests of all init containers that run before it). The reasoning is that init containers run one at a time and in order, so the peak demand is the heaviest prefix of the init sequence, while regular containers all run concurrently. A pod whose only large request is in a final init container that runs alone needs that much, not the sum.

The effective limit follows the same shape, with a further subtlety: if any container specifies a limit for a resource, the pod’s limit for that resource is the maximum of the per-container limits and the init sequence prefix maxima, and containers that did not specify a limit are treated as unlimited for that computation. A pod with one limited container and one unlimited container therefore has no effective limit, which is one reason the Guaranteed class requires every container to set limits.

Sidecar containers, which run for the pod’s lifetime alongside the main containers, count as regular containers for this arithmetic. Their requests add to the total and their presence can push a pod out of the Guaranteed class if they omit limits.

What the numbers cannot express

Requests and limits are per-resource scalars, so they cannot describe a workload whose need is a relationship between resources, and they cannot describe a workload whose need varies with time. A workload that needs 4 cores and 16 GiB simultaneously, but only ever 2 cores when using 8 GiB, is declared at its peak on both axes and occupies scheduling capacity it will never use concurrently. That is the cost of a scalar interface: it is trivially composable and it over-reserves.

The scheduler’s view is also a view of declarations, not of reality, so a cluster can be simultaneously over-committed in aggregate and under-utilised in practice, with both facts true and neither visible from the other. The measurement that closes the loop is usage data, which is a different subsystem with its own sampling interval and its own lag, and lag is precisely what makes the feedback controllers discussed in autoscaling feedback loops and stability hard to tune. What the scheduler does when the declared requests do not fit anywhere is covered in placement constraints and unschedulable states.