What a System Call Actually Costs

The claim that a system call costs “about a microsecond” is true often enough to be useless. The same instruction sequence costs a few hundred cycles when the call is trivial and the caches are warm, and several thousand when it touches a large structure, crosses into a slow device, or runs on a machine whose speculative-execution mitigations have made every kernel entry expensive. The interesting question is not the average but the decomposition: which part of the cost is fixed by the architecture, which is fixed by the kernel’s design, and which is a property of what the call was asked to do.

Splitting the cost that way also explains why optimisation efforts concentrate where they do. A mechanism that removes a third of a fixed cost is worth building only if the call in question does no real work; a mechanism that removes a copy is worth building whenever the copy is large. The two categories need different answers, and confusing them produces designs that make one call faster and the system slower.

The fixed part: entry, exit and the mode change

Between the syscall instruction and the first line of the handler, the machine does a small amount of work that cannot be avoided: it saves the return address and flags, loads the new instruction pointer, masks flags, swaps the GS base, loads a kernel stack, and pushes twenty-one registers’ worth of frame. On the way out it does the reverse, checks for pending work, and executes either sysret or the full iretq path. The instructions themselves number in the low hundreds.

The cycles they take are not the interesting number. The interesting number is the disruption. The syscall instruction is a serialising event for the front end: the pipeline’s speculative work on user instructions is discarded, the branch predictor has to learn a new target, and the return address stack is disturbed twice. An empty system call on x86-64 lands on the order of a few hundred cycles on hardware without entry mitigations, a figure that moves with microarchitecture rather than one that can be quoted as a constant.

Then come the mitigations. Kernel page-table isolation, introduced in response to the Meltdown attack, removes kernel mappings from the user page tables entirely and keeps a separate set for kernel mode. Every entry must therefore switch page-table roots, and every switch invalidates translations that are not tagged. The cost is paid on both directions of every system call, and it is not a few cycles: it is a TLB flush plus the refill of whatever the kernel touches next. Implementations reduce it with process-context identifiers where the hardware supports them, but the mechanism exists because of an attack, not because the design wanted it.

The same class of mitigation affects return: the fast sysret path cannot be used when the page-table root must change, so calls that would otherwise take the cheap exit take the expensive one. This is a concrete example of a general rule: the fast path is conditional on more than the architecture’s own constraints, and a security property can move a call from the cheap path to the expensive one without changing anything the caller can see.

The part that scales with the argument

A call that copies data has a cost proportional to the copy. read on a socket copies from kernel buffers to user memory; write copies the other way; stat copies a fixed-size structure out; recvmsg copies a variable-length scatter of segments. On the common architectures a cache line is 64 bytes, and a copy that stays inside L1 runs at a few bytes per cycle. A copy larger than the private caches runs at memory bandwidth, which is an order of magnitude slower per byte.

This is why the cost of a system call and the cost of the work it performs have to be considered together. A 1 KiB read is dominated by the trap; a 1 MiB read is dominated by the copy and the device. There is no single crossover point, because it depends on whether the source is cached, but the shape is consistent: small calls pay for the boundary, large calls pay for the data.

The copy is also where a class of security checks lives. Any pointer the caller supplies must be validated against the caller’s address space before it is dereferenced, and the validation is not a single comparison. It must handle the boundary between user and kernel ranges, the possibility that the address is valid but the page is not mapped, and the fact that the mapping can change under the kernel’s feet if another thread of the same process is running. The mechanism that does this is described in Copying Data Across the User-Kernel Boundary, and its cost is proportional to the number of pages touched, not the number of bytes copied.

Scheduling and accounting, which are not free either

A system call is an opportunity for the scheduler to run. The kernel checks, on the way out, whether a reschedule is needed: whether a higher-priority thread became runnable, whether the current thread’s slice expired, whether a signal is pending. That check is cheap when nothing is pending. When something is, the exit path becomes a context switch, and the cost jumps by the price of saving one thread’s state, restoring another’s, and presenting the incoming thread with caches full of somebody else’s data.

Accounting is a smaller but real addition. The kernel charges the time spent in the call to the process’s user or system time counters, which means sampling a clock source somewhere on the path. Reading a clock is itself a system-call-shaped problem: the fast source is a memory-mapped counter, and reading it is cheap, but on some hardware the counter is not synchronised across cores and the kernel must correct for the offset. Kernels therefore sample the clock where the accounting requires it rather than on every call, using counters that are cheap to update, but the accounting is never entirely absent.

There is a third cost that shows up only under concurrency: the structures the kernel touches on entry are shared. The per-CPU data is local, but the task structure, the file table, the memory map and any lock the handler takes are not. A syscall-heavy workload on many cores spends part of its time on cache-line transfers between cores, and that cost grows with core count even when the call itself is unchanged.

Comparing the entry mechanisms honestly

The three ways to enter the kernel have different fixed costs, and the differences are structural.

int 0x80 reads the gate in the interrupt descriptor table, switches stacks through the task-state segment, and saves a larger frame. syscall takes its target from a model-specific register and does none of that, which is why it was introduced. A sysenter/sysexit pair on 32-bit hardware was the earlier attempt at the same idea, with the same absence of a descriptor-table lookup, and it was tied to model-specific registers for its target and its return path. The vDSO goes further and removes the transition entirely for calls whose answer can be computed in user space, at the price of restricting what those calls may do.

Path	Entry cost	Exit	Typical use
syscall / sysret	one instruction, fixed entry address, no descriptor table	one instruction, no CR3 change	native calls, no isolation switch
syscall / iretq	as above	full frame pop, CR3 change allowed	calls needing a full frame or a table switch
int 0x80	descriptor-table lookup, TSS stack switch	iret	32-bit compatibility
vDSO	none	none	clock reads, getcpu

The table omits cycle counts deliberately, because they are not comparable across microarchitectures and because the ordering, not the magnitude, is what a design decision depends on.

Where the measurement goes wrong

Anyone who measures system call cost has to avoid three traps, and the traps explain most of the disagreement in published numbers.

The first is measuring a call that the kernel optimises away. Every call whose answer fits in a register and depends on nothing but immutable process state does essentially no work in the handler, so measuring it measures the boundary and nothing else. A figure derived from such a call should not be applied to read. The C library goes further on some architectures and answers a few of these calls from user space without entering the kernel at all, in which case the measurement is of the library, not the boundary.

The second is cache state. A loop issuing the same call repeatedly leaves the kernel’s data structures in cache, so the measured cost excludes the misses that a real workload pays. The difference between a hot and a cold call is often larger than the difference between two entry mechanisms.

The third is the clock. Timing a single call with a clock that has coarse resolution or that is itself a system call produces a measurement of the clock. The usual fix is to time a loop of many calls, which then includes the loop’s own overhead and the effect of the calls on each other. Neither approach is wrong; both measure something slightly different from what the caller wanted to know.

What survives all three is a qualitative picture. The boundary costs on the order of a few hundred cycles when the machine is cooperating. A call that copies a page and faults each page in costs substantially more, because a fault is a page-table operation rather than a memcpy. A call that blocks pays a context switch, and a call on a mitigated machine costs more than the same call on an unmitigated one by a margin that depends on the mitigation. These are the numbers that inform design: they say that avoiding a call is worth doing when the call does little, and worth ignoring when the call does a page of work.

The design consequence

Every mechanism in the rest of this cluster exists because of one of the costs above. The vDSO and System Calls That Never Trap removes the transition for calls that need no privileged work. io_uring and Shared-Memory Submission Queues amortises the transition over many operations and removes the copy where the hardware allows. Seccomp Filters and the Cost of Interception adds work to the fixed part, and the cost of that addition is why filter placement matters. Batched interfaces exist because the fixed part is fixed: if the boundary costs the same whether one operation or thirty-two cross it, crossing it once for thirty-two operations is the only way to make the per-operation cost small.

That is the whole shape of the trade. The kernel boundary is expensive per crossing and nearly free per byte, so designs either reduce the number of crossings, increase the work done per crossing, or remove the crossing for the queries that need no privileged decision. What none of them does is make a crossing cheap, because the crossing is what the hardware charges for changing privilege.