Page Migration: What a Move Between Nodes Costs

Changing a memory policy changes where future faults will place pages. It does not move the pages that are already there. For those, the kernel offers migration: a request to copy a page’s contents to a new node and rewrite every page-table entry that refers to it. The operation is available through mbind with MPOL_MF_MOVE, through move_pages, and implicitly through automatic balancing. It is much more expensive than a fault, and it can decline to do what was asked without reporting an error.

Why migration cannot be a pointer swap

A physical page is named by its frame number, and that frame number appears in every page-table entry that maps it: potentially thousands of entries, in many processes, across many CPUs’ TLBs. There is no indirection layer between a page-table entry and a frame, so moving a page means rewriting the entries.

The sequence the kernel follows for each page is constrained by that fact.

It begins by locking the page and checking whether it is movable at all. Anonymous pages in a normal mapping are movable. Pages that a device driver has pinned for DMA are not, because the driver holds a frame number that would become stale. A page that another process also maps is skipped by a move request unless the caller asks explicitly for shared pages to be moved as well, which is a separate flag and requires privilege. A page that is not movable is skipped, and the caller learns about it only if it asked for a report.

Next the kernel must remove the page from every page table that maps it. This is done by walking the reverse mapping, the structure that records for each page which address spaces map it, and replacing each entry with a migration entry. A migration entry is a special non-present encoding that marks the address as “in transition”, so a thread that faults on it while the copy is in progress waits rather than seeing stale data.

Then a new page is allocated on the target node, the contents are copied, and the reverse mapping is walked again to install the new frame number in each entry. Finally every CPU that might have cached the old translation must be told to invalidate it. That is a TLB shootdown: an inter-processor interrupt to each CPU in the set, with the initiating CPU waiting for acknowledgments. On a large machine the shootdown is often the dominant cost, because it is a synchronous message to every core that has ever touched the page.

The cost per page is therefore proportional to the number of mappings and the number of CPUs, not just to the 4 KiB copy. A single-page migration can be more expensive than the fault that created it by orders of magnitude, and migrating a range of a few gigabytes can take long enough that the workload’s phase has ended by the time it completes.

What MPOL_MF_MOVE asks for

mbind with MPOL_MF_MOVE sets the policy on the range and then attempts to migrate every page in it to satisfy the new policy. MPOL_MF_MOVE_ALL does the same but also includes pages mapped in other processes, which requires the caller to hold CAP_SYS_NICE.

Three outcomes are possible and they are not distinguished by the return value alone.

The call can succeed, having migrated every page. The call can return success having migrated only some pages, because some were not movable. And the call can return EIO, but only when the caller also asked for strictness: with the strictness flag alone the call fails if an existing page sits on a node the policy does not allow, and combined with a move flag it fails if some page could not be moved. The common case, a successful return with a partial migration, leaves the range with the new policy in force and a mixture of pages at old and new locations, a state that is easy to mistake for complete success.

The reason a page might not move is usually one of three: it is pinned by a device, it is a shared page whose other mapping the caller cannot reach, or the target node could not allocate a replacement and the allocation failed. The last case is the one that produces a partially migrated range on a nearly full node.

move_pages gives finer control. It takes an array of page-aligned addresses and reports, per page, either the node it is now on or a negative error code. That per-page report is the only way to know exactly what happened, and it is why the interface exists alongside mbind.

#include <numaif.h>
#include <stdio.h>

/* Move one page to node 1 and report where it ended up. */
int move_one(void *p)
{
    void *pages[1] = { p };
    int status[1] = { -1 };
    int nodes[1]  = { 1 };
    if (move_pages(0, 1, pages, nodes, status, 0) != 0)
        return -1;
    if (status[0] < 0) {          /* negative: errno-style failure code */
        fprintf(stderr, "page did not move: %d\n", status[0]);
        return -1;
    }
    return status[0];             /* the node the page is on now */
}

Passing a null pid means “the calling process”. Passing a node of -1 in the nodes array queries the current placement without moving anything, which is the supported way to ask where pages are without faulting them in.

Migration versus remapping

A different technique avoids the copy entirely for some cases. If a range has not been touched yet, its pages do not exist, and changing the policy is enough, and the faults that follow place the pages correctly. If a range is a private anonymous mapping that is about to be discarded, the pages can be dropped rather than moved.

For shared memory and file-backed pages, migration is the only option that preserves contents, and it moves the page for every mapping at once. That is either exactly what is wanted, when the goal is to relocate a shared region, or actively harmful, when one process migrates a shared buffer to suit itself and makes it remote for every other process.

The kernel’s own balancing uses migration in the other direction: it moves pages toward the node where the accessing task runs, rather than moving tasks toward their pages. The two approaches are complements, and which one a system uses is a policy choice with a measurable difference in behavior, described in Automatic NUMA Balancing and Its Failure Modes.

The cost of getting it wrong

Migration is irreversible in one practical sense: there is no record of where a page used to be, so a migration performed on a mistaken heuristic cannot be undone except by another migration with the same cost.

The failure that matters most is a feedback loop. If a page is migrated toward a task, and the task is then migrated toward a different page, the pair can chase each other. The kernel guards against this by rate-limiting migrations per task and by refusing to migrate a page that was recently migrated, but the guard is a heuristic and the loop is possible when a workload’s access pattern genuinely oscillates between nodes.

A second failure is migration under memory pressure. Migrating a page requires a free page on the target node. On a node that is nearly full, the allocation triggers reclaim, which may evict page-cache pages, which may cause the workload to re-read them from disk. A migration request that appears to be a placement optimization can turn into an I/O storm. The allocator’s fallback behavior and the reason a policy cannot always be honored are covered in Where a Memory Policy Attaches and How It Inherits.

A third is the interaction with huge pages. A transparent huge page is migrated as a single unit, which is cheaper per byte than 512 separate base-page migrations, but the entire 2 MiB must be movable. One pinned 4 KiB sub-page makes the whole huge page immovable, and the kernel’s response is either to skip the huge page or to split it into base pages first. Splitting is itself a costly operation that touches every mapping.

When to migrate deliberately

The clear case is a program that knows its data was placed wrongly and knows where it belongs: a service that initializes a large index at startup on whichever node happened to run the initializer, then wants it on a specific node before accepting requests. Migrating once, before the workload begins, costs a bounded amount of time and leaves the steady state correct.

The unclear case is a program that migrates in response to observed access patterns at runtime. The observation is always behind the access, the migration costs more than the faults it is trying to correct unless the range is reused many times, and the decision has to be made without knowing whether the pattern will persist. For anything but a long-lived, stable working set, leaving the pages where they are and accepting remote access is often cheaper than the migration that would fix it.