Readiness Notification and the Single-Threaded Event Loop

The thread-per-connection design is the obvious way to write a network server: accept a socket, hand it to a thread, let that thread call read and block. It works until the connection count grows. On Linux the default pthread stack is 8 MiB of virtual address space, most of it never touched, but ten thousand connections still mean ten thousand schedulable entities for the kernel to track, ten thousand stacks to reserve, and a run queue that the scheduler must walk. The memory is tolerable; the scheduling and the per-thread kernel bookkeeping are not, because at any moment almost every thread is asleep waiting for a socket that has nothing to say.

The event loop inverts that arrangement. Instead of one thread waiting on one descriptor, one thread waits on a set of descriptors and the kernel reports which members of the set have become ready. The thread wakes once per batch of events rather than once per event, and the number of threads is fixed by the number of cores rather than by the number of connections.

The kernel keeps the interest list

The naive interface for this is select, which takes three bitmaps of file descriptors and returns modified copies. Two properties limit it. The bitmaps are indexed by descriptor number, so the kernel must scan up to the highest descriptor in the set, and the constant FD_SETSIZE is 1024 on Linux: descriptors at or above it cannot be represented at all. Every call copies the bitmaps from user space to kernel space and back.

poll removes the fixed limit by taking an array of struct pollfd records, each carrying a descriptor and a requested event mask. The scan is still linear in the array length, and the array is still copied on every call, so the cost per call is proportional to the number of descriptors registered, not the number that are ready. For a server with ten thousand idle connections and a hundred active ones, that is a hundredfold waste.

epoll splits registration from waiting. epoll_ctl adds a descriptor to a set held inside the kernel, once, and the kernel stores it in a structure keyed by descriptor number. epoll_wait returns only the entries that are ready. The cost of a wait call is proportional to the number of ready descriptors, not the number registered, and no user-space array crosses the boundary. kqueue on the BSDs and macOS provides the same split with a different vocabulary: the program registers kevents, each naming a descriptor, a filter such as EVFILT_READ, and a flag word.

The essential consequence is that the runtime now holds a registry: descriptor to handler. The registry is the real data structure of an event loop, and every design decision downstream follows from it.

int ep = epoll_create1(0);
struct epoll_event ev = { .events = EPOLLIN, .data.fd = listen_fd };
epoll_ctl(ep, EPOLL_CTL_ADD, listen_fd, &ev);

for (;;) {
    struct epoll_event batch[64];
    int n = epoll_wait(ep, batch, 64, -1);   /* -1: block until something is ready */
    for (int i = 0; i < n; i++)
        dispatch(batch[i].data.fd, batch[i].events);
}

The batch size matters. Returning one event per call would pay the syscall cost once per event; returning sixty-four amortises it. The data field is a 64-bit union the caller may fill with anything, and most runtimes store a pointer to their own handler object there instead of a bare descriptor, which removes a hash lookup from the dispatch path.

Level-triggered reports state, edge-triggered reports change

epoll offers two notification disciplines, and the difference is the most common source of bugs in hand-written loops.

Level-triggered is the default. If a descriptor is readable when epoll_wait is called, it is reported; if the handler reads only part of the available data, the next call reports the descriptor again. The behaviour is easy to reason about because readiness is a property of the descriptor at the moment of the call. The cost is repeated wakeups: a descriptor left readable without being drained produces one event per loop iteration, which turns a slow client into a busy loop.

Edge-triggered, requested with EPOLLET, reports a descriptor only when its readiness changes from false to true. The kernel records that the transition has been delivered and will not deliver it again until the descriptor becomes unready and then ready once more. This is cheaper, one notification per transition rather than one per call, and it imposes a hard obligation on the handler: it must keep reading until read returns EAGAIN. A handler that reads one buffer’s worth and returns leaves data in the socket receive queue. The descriptor is still readable, but no new transition will occur, so no further event arrives and the connection hangs until the peer sends enough additional data to trip the threshold again.

The same obligation applies to writes. A non-blocking write may accept fewer bytes than requested; the remainder must be buffered by the runtime and the descriptor registered for EPOLLOUT, which is edge-triggered in the same way and requires writing until EAGAIN. This is why serious runtimes own an output buffer per connection rather than writing directly from the handler’s stack.

EPOLLONESHOT is a third option that disarms a descriptor after it fires, so that another thread must re-arm it explicitly. It exists to make a descriptor safe to hand between threads: without it, two threads waiting on the same set can both be woken for the same readiness and both attempt to read, and one of them blocks or gets EAGAIN unexpectedly.

Dispatch is a table, not a stack

The loop itself is small. It calls the wait primitive, walks the returned batch, and invokes a function per entry. There is no call stack per connection and no saved register set, because the handler runs to completion on the loop’s own stack and returns. That is the entire efficiency argument: a connection costs a handler object and a buffer, on the order of a few kilobytes, rather than a stack and a kernel task.

The argument also states the design’s central constraint. Every handler must return promptly. The loop has no mechanism to interrupt one, no timeslice, and no way to run two handlers at once. A handler that performs a blocking disk read, resolves a hostname through the system resolver, or computes a large matrix does not slow the loop down; it stops it. Every other connection waits for the duration. The loop’s cost per iteration is the sum of the handler durations, plus the wait syscall, plus whatever the kernel charges for the wakeup.

Runtimes mitigate this in three ways, each treated in its own place: by pushing genuinely blocking work onto a separate pool of threads, by making every operation that might block return a suspension point instead, and by running several loops on several threads with descriptors partitioned between them. The partition is usually done at accept time with SO_REUSEPORT, which lets multiple sockets bind the same port and have the kernel distribute incoming connections between them. A single shared listening socket with several waiters produces the thundering herd, where all waiters wake for one connection; EPOLLEXCLUSIVE reduces that by waking one waiter, and SO_REUSEPORT avoids it by not sharing the socket at all.

The costs that do not disappear

Readiness notification removes the per-connection thread, not the per-event work. Each event still costs a syscall to retrieve, a hash lookup or pointer chase to find the handler, the handler’s own system calls, and the copy of data between kernel and user space. A loop that handles a million events per second is doing a million dispatches, and at roughly a microsecond per system call the reads and writes alone would account for a full core’s worth of time before the loop’s own work is counted.

There is also a fairness problem with no natural solution. The ready list is returned in whatever order the kernel produces it, and a descriptor that is always ready, such as a socket receiving a continuous stream or a pipe being written faster than it is read, appears in every batch. Handlers for quiet descriptors are delayed by however long the busy ones take. Some loops address this by rotating the starting point of the scan or by capping the number of events processed per descriptor per iteration; others accept the imbalance because the busy connection is usually the one that matters.

Finally, the model is fundamentally about readiness, not about completion. The kernel says a descriptor can be read without blocking; it does not say how many bytes are available, and it does not perform the transfer. The program still issues the read, still handles partial results, and still maintains its own buffers. The alternative discipline, in which the kernel is handed a buffer and reports when the transfer is finished, produces a different runtime shape entirely, and is the subject of readiness and completion models compared. The obligation that every handler return quickly is the same obligation that cooperative scheduling generalises to tasks that are not handlers at all.