Copying Data Across the User-Kernel Boundary

A system call that takes a buffer takes a number. The kernel receives a pointer the way it receives any other argument: as a value in a register. It cannot use that value the way user code would. The address refers to the calling process’s address space, whose layout another thread of that process may be changing, and the page it names may not be mapped at all. On a machine where user mappings are absent from the page tables the kernel runs on, a plain load would fault; where they are present, a plain load of a user address would still have to be distinguished from a kernel access when it faults, and the fault handler has no way to know what the kernel was doing unless the access itself was marked.

Every byte that crosses the boundary therefore passes through a routine that does three things: translates the address into something the kernel can access, copies under a fault handler that knows how to recover, and reports how much was copied before anything went wrong. The routines that do this are among the most carefully written in the kernel, and their cost is the dominant term in any system call that moves data.

Why the pointer cannot be trusted

Three independent reasons, and any one of them is sufficient.

The address may be outside the process’s address space. A 64-bit kernel typically splits the address space so that user addresses occupy the lower half and kernel addresses the upper. A pointer with the high bits set names kernel memory; on a CPU that faults on supervisor accesses to user pages when the kernel asks the copy routine to check, the attempt is refused, but on one that does not, a copy routine that skipped the range check would read the kernel’s own memory at a location chosen by the caller. That is a direct read primitive and a direct write primitive if the call is a write-shaped one.

The page may not be mapped. User memory is demand-paged; a pointer into a valid range may refer to a page that has never been touched. The kernel’s access will fault, and the fault must be resolved by populating the page, which is legitimate for a user access and would be a bug for a kernel access.

The page may be protected. A pointer into a read-only mapping is fine for a call that writes from user memory and an error for a call that writes to it. The page tables encode this, but only the routine that knows the direction of the copy can interpret the resulting fault correctly.

There is a fourth reason that is architectural rather than logical: on many processors the kernel’s page tables do not contain the user mappings at all. Kernel page-table isolation removes them, so the kernel cannot access user memory without either switching page tables or using a mechanism the hardware provides to address the user space while in kernel mode. Both approaches exist; the second is cheaper because it avoids the switch, and it is the one used for the copy itself on architectures that provide it.

What the copy routine actually does

The straightforward implementation is a byte loop with a special annotation. The kernel marks the instructions that access user memory so that the fault handler can recognise a fault at those addresses, look up the recovery point, and resume at a label that returns an error instead of crashing. That mechanism, the exception table, is why a kernel function can safely dereference a possibly invalid pointer at all.

/* The essential shape: a copy that can fail part-way and must
   report how far it got. The access_ok check is separate and
   happens before any byte is touched. */
long copy_from_user(void *dst, const void __user *src, size_t n)
{
    if (!access_ok(src, n))
        return n;                 /* nothing copied; caller maps to EFAULT */

    return raw_copy_from_user(dst, src, n);  /* returns bytes NOT copied */
}

/* read(2) uses the residue to decide what to report. The kernel's
   copy_from_user already maps a nonzero residue to EFAULT, so the
   handler mostly has to turn the residue into the bytes moved. */
ssize_t do_read(int fd, void __user *buf, size_t count)
{
    long left = copy_from_user(kbuf, buf, count);
    if (left)
        return -EFAULT;           /* partial or total failure, per the kernel's choice */
    return count;                 /* all bytes moved */
}

Two details in that fragment carry most of the semantics. The residue convention, returning the number of bytes not copied rather than the number copied, is what makes partial failure representable without a second return value: the routine can stop wherever the fault occurred and say how much it left behind. The caller then decides what a partial copy means. For a read into a buffer that the kernel populated from a device, bytes already copied are on their way to being reported, and the residue is dropped; for a write from a buffer only partly readable, the direction reverses and the residue becomes the error.

The validation step and the copy step are separate because they answer different questions. access_ok checks that the range lies within the user address limit and that the arithmetic does not wrap. It cannot check that the pages are mapped, because checking would require walking page tables on every call, and the mapping can change immediately afterwards anyway. It is a range check, not a validity check, and its cost is a few comparisons.

The copy loop and why it is not memcpy

memcpy assumes both addresses are valid and that the regions do not overlap. Neither assumption holds. The copy routines are therefore specialised: they use the widest instructions available, they handle alignment by copying a few bytes until the destination is aligned, and they use non-temporal stores for large copies so that the data does not evict the caller’s working set from cache.

On x86-64 the widest useful instruction for a copy is a vector move of 16 or 32 bytes, and recent processors provide a fast string-copy instruction that the kernel uses for large transfers because it handles the loop and the alignment internally. The choice between them depends on size: below a threshold the setup cost of the string instruction dominates, above it the instruction wins. The thresholds are microarchitecture-specific and are selected at boot.

The alignment handling is not cosmetic. A 64-byte cache line is the unit of transfer between cache and memory, and a copy that crosses lines at an offset touches one more line than necessary at each end. For a copy of a few hundred bytes the effect is a few percent; for a copy of a page it is negligible; for a copy of a single small structure it can be the difference between one cache line and two, which is why some interfaces are laid out so that the structure fits in a line.

Non-temporal stores matter for the opposite reason. A copy of a megabyte through the cache evicts whatever the calling thread was using, so the thread returns to user mode with a cold cache. Streaming stores bypass the cache and write directly toward memory, which is slower per byte for small copies and faster in aggregate for large ones because it does not disturb the rest of the working set.

The page-fault path, which is where the cost hides

The copy routine’s instruction count is not the cost of a large transfer. The cost is the page faults.

Each page the kernel touches that is not resident produces a fault, and the handler must decide what kind of fault it is: a legitimate user-page fault that the kernel should resolve by allocating and zero-filling, a permission violation that must become EFAULT, or a kernel bug that must crash. The decision is made by consulting the exception table entry for the faulting instruction and by checking whether the address is within the user range and whether the access direction matches the mapping’s permissions.

Resolving a fault is expensive in a way that a copy is not: it may require allocating a page, zero-filling it, updating page tables, and invalidating a translation. On the order of a microsecond per fault is the right order of magnitude, against a few tens of nanoseconds for the copy of the same page once it is resident. A call that copies a megabyte of freshly mapped memory therefore pays for 256 faults, and those faults dominate everything else the call does.

This is why pre-faulting a buffer before a large transfer can be worth doing, and why a program that allocates a buffer and immediately passes it to a system call is paying for the pages twice — once in the fault handler and once in the copy. It is also why the first call on a buffer is much slower than subsequent ones, a difference that shows up in any measurement that does not warm up.

Zero-copy, and what it actually avoids

The phrase describes removing the copy between kernel and user memory. What remains, in every real implementation, is the pinning: the pages must be made resident and their physical addresses recorded so that a device can write to them directly. Pinning costs a page-table walk per page and prevents the page from being swapped or migrated for the duration, which is a real constraint on the memory manager.

Where the hardware supports it, the copy can be avoided entirely by mapping the same physical pages into both address spaces. A shared ring buffer is the usual construction: the kernel and the process agree on a region, both map it, and data crosses without a copy because both sides read the same memory. The cost moves from the copy to the synchronisation: the two sides must agree on who writes what and when, and the ordering rules that make that safe are stricter than anything a copy requires. The mechanism built on this idea is described in io_uring and Shared-Memory Submission Queues.

For network receive in particular, the kernel can hand a page to the process rather than copying out of it, and the process returns the page when done. That trades a copy for a page-ownership protocol and a bounded pool, and it fails when the process holds pages longer than the pool can tolerate.

What the boundary costs in practice

The honest summary is that copying is cheap per byte and expensive per page fault, and that the validation around it is cheap per call. A read of 4 KiB from a warm buffer costs a few thousand cycles, most of which is the trap and the copy. The same read into a cold buffer costs an order of magnitude more because of the fault. The same read into a buffer the kernel has to bounce, because the device cannot address it or because the mapping is not what the hardware expects, costs another copy on top.

The design pressure is therefore toward fewer crossings and fewer faults, not toward faster copy loops. The copy loop has been optimised for decades and is close to memory bandwidth; the faults have not, because resolving a fault is inherently a page-table operation. An interface that moves a large object in one call and touches each page once is near the floor. An interface that moves the same object in many small calls pays the fixed cost repeatedly, which is the argument developed in What a System Call Actually Costs.