Footprint Under a Container Limit: Reclaim, Working Set, and the OOM Decision

A process whose resident size is below its container limit can still be killed for exceeding it. The reason is that the limit applies to a different quantity than RSS: it applies to the memory charged to a control group, which includes page cache the group faulted in and kernel memory allocated on its behalf. A process that reads a large file has a large charge and a modest resident set; a process whose anonymous memory is untouched has a small charge and a large address space. Reading the cgroup’s own counters is the only way to know which situation is in effect.

What the memory controller charges

The memory controller in the unified hierarchy accounts memory to a cgroup by charging each page to the group that first faults it in, and to the group that owns the process that allocates kernel objects on the group’s behalf. The charged categories are:

• Anonymous pages, including shared anonymous and the private copies made by copy-on-write.
• Page cache pages, both file-backed mappings and the page cache populated by reads and writes through the filesystem.
• Kernel memory, including slab objects accounted to the group and the pages of page tables and kernel stacks used by the group’s tasks.

The counters expose these separately, and the separation is the point. A group near its limit with a large page-cache component is not in danger: the kernel reclaims clean page cache before it considers killing anything. A group near its limit with a large anonymous component is in danger, because reclaiming anonymous memory requires swap, and a group with no swap configured has nothing to reclaim.

Two further distinctions matter and are less obvious:

The cache is not free. Page cache charged to a group counts against the limit even though it is reclaimable. A workload that streams a large file through a group will see the group’s charge rise to the file’s size, and the kernel will reclaim the cache under pressure rather than fail the allocation. The cost is repeated re-reads, which is a performance problem, not a failure.

The cache is not always reclaimable. Pages under writeback, pages whose writeback cannot complete because the storage is unavailable, and pages pinned by a device or by a kernel operation cannot be reclaimed immediately. A group whose charge is entirely dirty page cache cannot free anything until writeback completes, and during that window an allocation can fail.

Why the killer picks the group it picks

When a group cannot satisfy an allocation after reclaim, the kernel invokes the out-of-memory killer for that group. The selection is by a score computed per process, historically called the badness score, combining the process’s resident size with an adjustment that the administrator can set per process. The process with the highest score is killed, and the group’s charge falls by its resident size.

The consequences for footprint work are practical:

A large process is killed before a small one, regardless of which one grew. A sidecar that has been running since the group started with a large but stable footprint can be the victim of a leak in a small process that pushed the group over the limit. The killer’s choice is about who frees the most memory fastest, not about who caused the problem.

Killing a process does not necessarily free what the group is charged. A process that has exited but whose memory is still mapped by another process, through a shared mapping or a page cache page it faulted in, leaves its charge in place. The group may remain at its limit after the kill, and the killer may fire again.

The group limit is enforced at allocation time, not continuously. A group can sit exactly at its limit indefinitely if nothing allocates. The failure appears at the first allocation that cannot be satisfied, which is often far from the code that caused the growth.

The counters that answer the question

The controller exposes a set of per-group files that together describe the state well enough to diagnose a limit problem. The exact names have been stable across recent kernel versions, and each answers a specific question:

• Current charge. The total bytes charged to the group. Comparing this against the limit is the direct measure of headroom.
• Peak charge. The high-water mark since the group was created or since the counter was reset. This is the number that tells whether a group ever came close to its limit, which is invisible from a current-charge sample taken every minute.
• Anonymous, file, and slab breakdown. Splits the charge by category. Anonymous is the unreclaimable-without-swap portion; file is the page cache portion; slab is kernel objects.
• Reclaim activity. Counters for pages scanned, pages reclaimed, and the number of times the group hit its limit and had to reclaim. A group whose reclaim counters climb steadily is operating at its ceiling, even if its current charge looks comfortable.
• Kill count. The number of processes belonging to the group that an OOM killer has killed. The counter survives the death of the process that caused it, but it is hierarchical: a nonzero value can come from a descendant group whose own limit was hit rather than from this group’s limit, and the non-hierarchical variant of the same file is what separates the two.
• Events. Counters rather than measurements: how many times the group hit its hard limit, how many times it went over its high watermark and allocation was throttled, and how many processes were killed. Reading these at a fixed interval is cheaper than tracking the charge itself, and a count that keeps rising is the signature of a group living at its ceiling.

The most useful pair is peak charge against anonymous charge. If the peak is close to the limit and the anonymous component is small, the group is being pushed by page cache and the fix is either a larger limit or a change in I/O pattern. If the anonymous component is close to the limit, the group is holding real memory and the fix is in the program.

# Charge breakdown for one cgroup, in a cgroup v2 hierarchy.
CG=/sys/fs/cgroup/example.service
awk '{ printf "current %.1f MiB\n", $1/1048576 }' "$CG/memory.current"
awk '{ printf "peak    %.1f MiB\n", $1/1048576 }' "$CG/memory.peak"
while read -r key val; do
    printf "%-10s %8.1f MiB\n" "$key" "$(awk -v v="$val" 'BEGIN{print v/1048576}')"
done < <(grep -E '^(anon|file|slab|kernel) ' "$CG/memory.stat")

The same statistics exist per-process in the process’s own status output, which reports the resident set, the peak resident set, and the sizes of the various mapping categories. The per-process numbers and the group numbers do not sum: shared pages are charged to one group but resident in several processes, and kernel memory charged to a group has no per-process representation at all.

Reclaim and the working set

The kernel’s notion of a group’s working set is the memory it has touched recently enough that reclaiming it would cost more than keeping it. The page cache uses a two-list approximation of this: pages referenced once sit on an inactive list, pages referenced again are promoted to an active list, and reclaim takes from the inactive list first. A page that is evicted and immediately re-read returns to the inactive list and must be referenced again to be promoted.

For a group under a limit, the practical effect is thrashing: if the working set exceeds the limit, the kernel evicts pages that the workload will need again, and the cost appears as I/O rather than as allocation failure. A group can run indefinitely in this state, far over its useful working set and far under its limit in terms of failure. The symptom is throughput collapse with no OOM kill and no obvious memory number out of range, which makes it the hardest container memory problem to attribute.

Detecting it requires the reclaim counters rather than the charge: a group whose reclaim rate is high relative to its I/O rate is thrashing, and the fix is a limit that accommodates the working set rather than one that accommodates the live set.

Limits that are not the memory limit

Three other limits produce failures that look like memory exhaustion and are not.

The swap limit. A group with a memory limit and a swap limit can exceed the memory limit by using swap, up to the swap limit. A process whose anonymous memory is mostly swapped has a small charge and a large live set, and its performance is dominated by swap-in. Disabling swap for a group makes anonymous memory strictly bounded, which converts a latency problem into an allocation failure.

The pid limit and the file descriptor limit. Neither is memory, but both cause allocation failures inside the kernel that surface as errors a program may report as memory exhaustion. A fork that fails because of a pid limit is not an out-of-memory condition, though the error code may be the same.

The kernel memory limit. Kernel objects allocated on behalf of a group, such as socket buffers or dentries created by the group’s path lookups, are charged to the group and, on some configurations, bounded separately. A workload that opens many sockets can hit a kernel memory limit while its user-space footprint is small.

Connecting a group’s charge to the program’s behaviour

The bridge between a cgroup number and a program’s allocation behaviour is the same mapping breakdown used for a single process, applied at the group level. A group whose anonymous charge grows while its file charge is flat has a program retaining memory; a group whose file charge grows while anonymous is flat has a program reading more data than its limit can cache. The two have entirely different fixes, and the cgroup counters alone cannot distinguish which program in the group is responsible. That requires the per-process mapping attribution and, for the anonymous portion, an allocation trace.

A slow growth in anonymous charge is the case where the group-level view is least informative and the program-level view is essential. The group shows a gradual rise with no single event to point at, and the only way to find the cause is to compare live sets over a long interval, which is the long-running growth analysis that operates below the container layer.