First-Touch Placement and the Faulting Thread

A freshly allocated anonymous page has no node. It has an address in a virtual range and a page-table entry that is not present. For anonymous memory the rule is stated precisely: an initial read is served from a shared zero page that the kernel keeps for the purpose, and the page is allocated and placed only when the thread writes to it. The physical page behind it is therefore chosen at the first write, and the kernel chooses it based on which CPU that thread is running on at that instant. This one rule explains most of the surprising placement behavior on NUMA machines, including the case where a single-threaded initialization phase spreads an entire working set across every node and leaves the parallel phase reading remote memory.

The default is not “where the allocator ran”

Consider the obvious naive model: malloc returns a pointer, and the allocator decides where the memory lives. That model is wrong for any allocation large enough to be served by mmap, which on glibc means allocations at or above the default mmap threshold of 128 KiB. For those allocations, malloc asks the kernel for a range of virtual address space and nothing else. The kernel reserves the range, records it as a virtual memory area with an associated memory policy, and returns. No physical page exists.

For smaller allocations, the page may already exist, having been faulted in earlier by whatever thread grew the heap, and the placement is inherited from that earlier fault. The distinction matters: large allocations are placed lazily and per-page, while small ones are placed by history.

The first-touch rule is a consequence of a design constraint rather than an optimization. The kernel cannot know at mmap time which thread will use the memory, and pre-faulting on behalf of the allocating thread would place every page on that thread’s node. Deferring the decision to the fault costs one fault per page but produces placement that follows use.

Inside the fault

The path from a store instruction to a physical page has several decision points, and the policy hooks sit at each of them.

The store misses the TLB, the hardware page walker finds no valid entry, and the CPU raises a page fault. The kernel’s fault handler looks up the virtual memory area containing the address. That area carries a mempolicy reference, which may be null, in which case the task’s policy applies, and if that is null too, the system default applies. The effective policy is resolved before any page is allocated.

Then the allocator is called with a preferred node derived from the faulting context. Under the default policy, the node is the one the faulting CPU belongs to, obtained from the per-CPU variable that records the current node. Under a MPOL_BIND policy, the allocator restricts itself to the nodes in the nodemask and picks the closest one to the faulting node. Under MPOL_PREFERRED, it tries one node and falls back to the local node. Under MPOL_INTERLEAVE, it ignores the faulting node entirely and takes the next node in a round-robin sequence stored in the task’s policy state.

If the chosen node cannot satisfy the order, because its free lists are empty, the allocator walks the node’s zonelist in distance order and takes the first zone with pages available. That fallback is the reason a machine can appear to violate its own policy under memory pressure without any bug being present.

Why the faulting CPU is the right answer and the wrong one

The rule assumes that the thread that first touches a page is representative of the threads that will use it later. For a single-threaded program, it is exactly correct: the thread that faults the page is the thread that will read it, and the page is on that thread’s node.

For a program that initializes data in one phase and processes it in another, the assumption breaks completely. A common shape is a serial loop that fills an array before spawning workers. Every page of the array is faulted by one thread on one node, and every worker then reads from that node. If the array is larger than the local node’s capacity, the pages spill onto other nodes by fallback, so the array is scattered according to which node had free memory at the moment of each fault, which is not a pattern anyone chose.

The failure is not subtle in its consequence: the parallel phase becomes bandwidth-bound on the link that feeds the single node holding the data. The fix is to make the initialization phase parallel, so that each thread faults the pages it will later process, or to change the policy for that range.

Policies that replace the default

Linux exposes several policies through mbind and set_mempolicy. Each is a different answer to the question “which node should this page come from”.

MPOL_DEFAULT means the faulting node, with fallback by distance. It is the policy in force when nothing has been set, and it is what makes first-touch work.

MPOL_BIND restricts allocation to the nodes in a mask, preferring the one closest to the faulting node. It is a hard constraint in the sense that the allocator will not allocate outside the mask even under pressure; instead, it will reclaim within the mask or fail the allocation. This is the policy for a data structure whose size is known to fit, and whose placement must be predictable.

MPOL_PREFERRED names a single node to try first and falls back to the faulting node if that node is full. It is the weakest of the explicit policies and the least disruptive when the preferred node runs out.

MPOL_INTERLEAVE distributes pages round-robin across the nodes in the mask, choosing each page’s node from its index within the mapping, so the distribution depends on the address rather than on which thread faults first. This is the policy for aggregate bandwidth, discussed in Interleaving Pages Across Nodes for Aggregate Bandwidth.

A weighted variant of interleave takes a per-node weight array instead of treating all nodes equally, so a range can be spread proportionally rather than uniformly. The weights are relative, and the implementation accumulates them to decide each successive page’s node. Kernels that do not support the mode reject it with EINVAL, which is how support is detected.

#include <numaif.h>
#include <numa.h>

/* Bind the range to node 0 only; fail loudly if the kernel refuses. */
int bind_to_local(void *addr, size_t len)
{
    struct bitmask *mask = numa_allocate_nodemask();
    numa_bitmask_setbit(mask, 0);
    int rc = mbind(addr, len, MPOL_BIND, mask->maskp,
                   mask->size + 1, MPOL_MF_MOVE);
    numa_free_nodemask(mask);
    return rc;
}

The MPOL_MF_MOVE flag is what makes mbind affect pages that already exist: without it, the call only changes the policy for future faults, and a range that has already been fully touched keeps its original placement. With it, the kernel attempts to migrate the existing pages to satisfy the new policy, which is a much more expensive operation and one that can fail partway.

Policy inheritance across fork and exec

Two details in the policy model surprise people.

The first is that policies are attached to address ranges, not to allocations. Splitting a virtual memory area, whether by mprotect, by munmap of a subrange, or by mremap, can split the policy association, and the pieces inherit the policy that was in force. Merging adjacent areas with different policies is refused, so a program that repeatedly sets and clears policies on neighboring ranges accumulates small areas and eventually exhausts the mapping limit.

The second is inheritance. A child created by fork or clone inherits the parent’s memory policy, so pages it touches are placed according to the parent’s settings even if the child runs on a different node. The task policy also survives execve, because it belongs to the task and not to the address space: a launcher can set a policy and then start a program that contains no policy code of its own, and the program runs under the inherited policy. Policies attached to address ranges are the opposite case, since they describe regions of an address space that execve discards, so a range policy cannot outlive the image that installed it.

Shared memory is the third case. Pages of a shared memory segment belong to the segment, not to any process, so the first process to fault a page fixes its node for every process that maps the segment afterward. Two processes on different nodes sharing a buffer will have one of them accessing remotely, and moving the page to suit one moves it for both.

The consequence is that first-touch placement on shared memory is a race whose outcome depends on scheduling order. Programs that care about it either fault the whole shared region from the intended node before starting workers, or accept that placement is arbitrary and design the access pattern around it.

Diagnosing where the pages went

The kernel reports per-node page counts per process. Reading /proc/<pid>/numa_maps yields one line per virtual memory area, carrying the area’s start address, the policy in force with its node mask, and a per-node count of the pages currently mapped into it.

The output distinguishes the cases that matter. A large anonymous area with all its pages on node 0 and a policy of default means a single thread faulted it. The same area with pages spread across nodes and a policy of interleave an=0-1 means the policy did its job. An area whose policy says bind:0 while its counts show pages on node 1 is the signature of a policy set after the pages were touched, or of an allocation that fell back under pressure.

The counts are the useful part; the policy field is the explanation. Comparing the two is the fastest way to tell whether placement is what was intended, and the counters that turn a placement problem into a rate are described in Measuring Remote Access with Hardware Counters. The topology those node numbers refer to is described in Reading NUMA Topology from the SRAT and SLIT.