Ordered Shutdown, Exit Handlers, and What the Kernel Reclaims

Process exit is the one cleanup path that is guaranteed to run and guaranteed to be incomplete. The kernel will release every descriptor, unmap every region, and discard the address space, and it will do so without consulting the program. What it will not do is flush a buffer the program was holding in user space, complete a protocol handshake, or remove a lock file. Those obligations fall to exit handlers, and exit handlers have a contract narrow enough that most programs that rely on them for correctness are relying on something the contract does not provide.

The sequence a normal exit performs

A call to exit does not terminate the process immediately. The C standard specifies the order: functions registered with atexit are called in reverse order of registration, then streams are flushed and closed, then files created by tmpfile are removed, and then control passes to the host environment. The status passed to exit becomes the process’s exit status, truncated to the low eight bits as observed by the parent’s wait.

C++ adds a layer: objects with static storage duration are destroyed, and the destructors run before or interleaved with the atexit handlers depending on the implementation’s choice. The standard permits an implementation to destroy statics either before or after the atexit functions it registered, so a handler that touches a static object may find it already destroyed. This is why the ordering guarantee for statics is stated only within a translation unit and only among objects whose construction order is known.

_exit and _Exit skip all of it. They terminate the process and let the kernel do the reclamation. The distinction matters because _exit is the only safe way to terminate from a forked child that has not called exec: the child inherited the parent’s atexit handlers and its buffered streams, and running them would flush the parent’s buffers a second time and run the parent’s cleanup against the child’s state. That is a duplicated-output defect that appears whenever a program writes to a buffered stream, forks, and the child exits with exit.

Why the handler list is a stack

Reverse order of registration is the same LIFO rule as scope exit, applied at process scale, and it exists for the same reason: a handler registered later may depend on a facility set up by a handler registered earlier. A program that initialises a logging subsystem, then a configuration store, then a database connection pool registers its shutdown in the opposite order, so the pool closes while the config and logging are still usable.

The analogy to scope exit ordering stops at one important point: the exit handler stack has no unwinding. There is no exception, no error return, and no way to signal failure. A handler that cannot complete its work has nowhere to report that. POSIX specifies that a handler registered with atexit is called with no arguments and returns nothing, and if it terminates by calling exit or longjmp, the behaviour is undefined.

That constraint shapes what belongs in a handler. Flushing a buffer and closing a descriptor belong, because they cannot meaningfully fail in a way the program could act on at that point. Waiting for a network peer to acknowledge a shutdown message does not belong, because it can block for an unbounded time and the handler cannot report that it gave up. Sending a message with a timeout belongs only if the timeout is short and the failure is acceptable.

The number of handlers is also bounded. POSIX requires that at least 32 handlers can be registered, and the C standard sets the same floor; an implementation may allow more, and the limit is discoverable only by observing the failure of atexit, which returns a nonzero value when the registration cannot be stored. A library that registers a handler per instance can exhaust the limit silently if it ignores the return value.

Draining before the exit path begins

The structural answer to the exit handler’s limitations is to not put the important work there. Instead, the program’s main loop performs an ordered shutdown before it calls exit, and the handlers are left to cover the case where the ordered shutdown did not happen.

A shutdown sequence for a server has a shape that is determined by dependency, not by preference. The steps are:

1. Stop accepting new work, by closing the listening socket or by removing the process from the load balancer.
2. Signal the workers that they should finish, and wait for the in-flight requests to complete, bounded by a deadline.
3. Flush and close the resources the workers used, in reverse order of their acquisition.
4. Release the global resources: the connection pool, the metrics exporter, the log file.
5. Return an exit status that reflects whether the drain completed within the deadline.

Step 1 comes first because a listener that stays open while workers are draining will keep queueing new connections that no one will serve. Step 2 needs a deadline because a worker blocked on a slow client will otherwise hold the process open indefinitely, and the deadline is what turns a hang into a reported failure. Step 3 is where the ordering discipline from the rest of this cluster applies; step 4 is where a bulk release such as an arena or a pool close collapses many individual releases into one operation.

The signal that starts this sequence is itself a resource. SIGTERM is the conventional request to terminate, and its default action is to terminate the process without running handlers at all. A program that wants an ordered shutdown installs a handler that sets a flag, and the main loop checks the flag. SIGINT from a terminal has the same default action. SIGHUP is conventionally a reload, and its meaning is entirely up to the program.

The handler must not do the shutdown itself. It runs in a context that interrupted arbitrary code, and only the async-signal-safe functions may be called from it, as the cancellation discussion in cancellation and half-finished state notes. Setting a volatile sig_atomic_t flag or writing to a self-pipe is the whole of the permitted work.

What the kernel does without being asked

When the process finally exits, whether through exit, _exit, or a fatal signal, the kernel releases everything the process held: the descriptor table, the address space and its mappings, the interval timers, the signal dispositions, the pending signals, and the process’s entry in the scheduler. The memory is reclaimed regardless of whether the program freed it, which is why a short-lived program can exit without freeing its heap and still be correct.

Three qualifications keep that guarantee from being a licence to ignore cleanup.

The first is that kernel reclamation covers kernel state only. Shared memory segments, POSIX named semaphores, file locks, and SysV IPC objects are not owned by the process in the same way. A named semaphore created with sem_open persists after the creating process exits and must be unlinked explicitly; a file lock is released when the descriptor closes, which the exit does, but the file’s contents may be left in a state the next reader cannot interpret. A lock file left on disk is the classic case: the kernel closes the descriptor, and the file remains, and every subsequent start sees a stale lock.

The second is that the reclamation is not atomic with respect to other processes. A descriptor closed at exit is closed at exit, and a peer that was mid-transfer sees a reset rather than an orderly close. Data that was written and not fsynced is in the page cache and will be written back eventually, but the ordering relative to the peer’s read is not controlled by the exiting process. That is why the drain step precedes the exit rather than being replaced by it.

The third is that reclamation happens at exit, not at any earlier point. A process that leaks a descriptor per request for a week and then exits cleanly has leaked for a week. The kernel’s guarantee is about the end state, not the trajectory, and the trajectory is what leak detection measures.

The exit status as a resource report

The exit status is the only channel through which a terminated process can report whether its cleanup succeeded, and it is eight bits wide. The convention that zero means success and nonzero means failure is universal; the meaning of specific nonzero values is up to the program, except for the values above 128 that shells use to report death by signal.

A shutdown that timed out waiting for workers should not report success. Distinguishing the two outcomes in the status is what allows a supervisor to decide whether to restart, and it is why the drain deadline belongs in the program rather than in the supervisor: only the program knows which of its resources were left in an intermediate state.

The signal case has no status at all in the ordinary sense. A process killed by a signal is reported by the parent as terminated by that signal, and the exit status is not set. Nothing in the program ran, no handler was invoked unless the signal was caught, and no cleanup occurred. The kernel did its reclamation, and everything else was skipped. That is the boundary of the guarantees in this cluster: deterministic cleanup covers the paths where the program is still executing, and the kernel covers the paths where it is not.