Requests, Limits, and the Two Different Jobs They Do

A container specification carries two numbers per resource, and they are not two views of the same quantity. The request is a claim made to the scheduler: this container needs this much, so place it only on a node with that much unreserved capacity. The limit is a ceiling enforced by the kernel: this container may not use more than this, and if it tries, it is slowed or killed. Setting them equal is a common default and a reasonable one, but treating them as interchangeable is the source of two distinct failure modes that look nothing alike. Neither number describes what the container actually receives, because the host’s own scheduling sits above both.

The request is a scheduling input

The scheduler’s problem is to assign pods to nodes such that the sum of requests on each node does not exceed the node’s allocatable capacity. It does not look at actual usage, because actual usage is not known at scheduling time and is not stable. It looks at the declared requests of the pods already bound to the node, subtracts them from allocatable, and checks whether the incoming pod’s requests fit.

This is why a cluster can report low CPU utilization while refusing to schedule new pods. The node’s allocatable capacity might be thirty-two cores; if the pods on it have declared requests summing to thirty-two cores, no further pod fits regardless of the fact that those pods are using four cores between them. The scheduler’s view is a reservation ledger, and the ledger is full. Utilization metrics measure the other thing.

The allocatable figure is itself smaller than the machine’s capacity. The kubelet reserves a portion for the operating system and for the container runtime, and it may reserve more for eviction thresholds. On a node with a large core count the difference is small in relative terms; on a small node it can be a meaningful fraction, and a workload sized to the machine’s nominal capacity will not fit.

Requests also determine the quality-of-service class, which determines what happens under pressure. A pod whose containers all declare requests equal to their limits for every resource is guaranteed. A pod that declares no requests at all is best-effort and is the first candidate for eviction. A pod in between is burstable, which is the majority of real workloads and the class whose behaviour is least intuitive.

The limit is a kernel enforcement point

The mechanism behind a CPU limit is bandwidth control in the completely fair scheduler. The kernel maintains a per-cgroup quota and period: a container may consume up to its quota within each period, and when the quota is exhausted the container’s threads are throttled until the next period begins. The period is a global tunable measured in microseconds, and the quota is derived from the limit divided by the period.

The consequence is that a container at its limit is not slowed smoothly. It runs at full speed until the quota is exhausted, then stops until the period boundary. A container that needs slightly more than its quota finishes its work in a burst and then waits, and the wait is visible as latency, not as reduced throughput. The average utilization over a period can look correct while individual requests see a stall of up to one period.

The stall length has a specific upper bound: one period, since the quota refills at the boundary. With the default period of a hundred milliseconds, a throttled container can be stopped for up to a hundred milliseconds at a time. For a batch job that is irrelevant. For a request-serving container whose latency budget is tens of milliseconds, a single throttle event can consume the entire budget, and a container that is throttled on most periods will show a p99 latency that is roughly its p99 service time plus a period.

Throttling is reported per cgroup as a count of throttled periods and a cumulative throttled duration. Those counters are the only direct evidence of the mechanism. High CPU utilization without throttling means the limit is not binding. Moderate utilization with heavy throttling means the workload is bursty relative to the period and the limit is too tight for its burst shape even though its average fits.

resources:
  requests:
    cpu: "500m"      # scheduler reserves half a core on the node
    memory: "512Mi"
  limits:
    cpu: "500m"      # CFS quota: 50 ms of runtime per 100 ms period
    memory: "512Mi"  # exceeded -> OOM kill, not throttling

The CPU and memory limits are enforced by entirely different mechanisms, and this is the single most useful thing to understand about the pair. CPU is a rate resource: exceeding the limit means running slower. Memory is a quantity resource: exceeding the limit means the allocation fails. The kernel cannot make a page allocation take longer, so the cgroup’s memory controller invokes the out-of-memory killer against the container when the limit is reached and reclaim cannot free enough.

Memory limits and the eviction distinction

A container that exceeds its memory limit is killed by the kernel’s OOM killer, and the pod’s restart policy determines what happens next. The kill is immediate and does not give the process a chance to handle the condition. A container that stays under its own limit but pushes the node toward its eviction threshold is a different case: the kubelet evicts pods to reclaim node capacity, and it ranks candidates by QoS class and by how far their usage exceeds their requests.

The distinction matters because the two events have different signatures. An OOM kill produces a container exit with a specific status and a restart count increment on one container. A node-pressure eviction produces a pod eviction event and the pod is rescheduled elsewhere. The first is a statement that the container’s own limit was wrong; the second is a statement that the node’s aggregate requests were understated relative to actual demand.

A pod with no memory request is the most exposed. It contributes nothing to the scheduler’s ledger, so it can be packed onto a node that is nominally full, and it has no claim to defend when the node comes under pressure. Its usage is invisible to the scheduler and its eviction is the kubelet’s first resort. Setting a memory request is therefore not about performance; it is about being counted.

Memory requests also affect the OOM score. The kernel assigns each process an out-of-memory score, and the container runtime adjusts it so that containers using more memory relative to their requests are more likely to be chosen. A container that sets a small request and uses far more is therefore a preferred victim, both for the kubelet’s eviction ranking and for the kernel’s own selection within the cgroup hierarchy.

Why equal requests and limits are not always right

Setting request equal to limit produces the guaranteed class and removes the gap between what the scheduler reserved and what the kernel permits. It is the correct choice for workloads whose resource needs are known and stable, and it makes capacity planning arithmetic exact: the sum of limits is the sum of requests, and the node is either full or not.

The cost is lost burst capacity. A workload that idles at a tenth of a core and bursts to two cores needs a request of two cores under this scheme, which wastes 1.9 cores of scheduling capacity on every node it lands on. The alternative is a low request and a high limit: the pod schedules cheaply and bursts into whatever capacity is left over on the node. This is the burstable class, and it works exactly as intended until every pod on the node tries to burst at once.

The failure mode of overcommitted burstable pods is not throttling, because their limits are high. It is contention: the node’s physical cores are shared, each pod gets a smaller share of actual CPU than it expects, and latency rises across the board without any container hitting a limit. The scheduler cannot prevent this because it was told the pods needed little. This is the trade in its purest form: the request buys predictable performance by consuming scheduling capacity, and the limit buys utilization by permitting contention.

The practical resolution is to size requests from a measured percentile of usage rather than from the peak, to size limits from the peak with headroom, and to accept that the difference between them is a bet that not all pods burst simultaneously. For workloads where that bet is unacceptable, the guarantee has to be bought explicitly by raising requests, which is the same thing as buying capacity.

The compound effect on scaling

Horizontal pod autoscaling reads utilization as a percentage of requests, not of limits. A deployment with requests set low will report high utilization at modest absolute usage and will scale out earlier; a deployment with requests set high will report low utilization and scale later, or not at all. The autoscaler’s target is therefore a function of the request, and changing requests changes the scaling behaviour without changing the workload.

This coupling is a frequent source of oscillation. Raising requests to improve scheduling predictability lowers reported utilization, the autoscaler removes replicas, the remaining replicas see higher load, and utilization climbs again. The loop settles only when the target and the requests are chosen together. The autoscaling control loop describes the damping that prevents this from becoming a permanent oscillation, but damping does not fix a target that was computed against the wrong denominator.