fork and exec: Which Descriptors Survive and Why

Starting a program on Unix is two operations, not one. fork produces a second process that shares the first one’s memory as copy-on-write and its descriptors as references to the same open file descriptions. execve then replaces the image, discarding the address space and every mapping the old program had built. The descriptor table is not discarded. It is the one piece of process state that survives the transition, minus the entries marked to be closed, and that asymmetry is where a large class of leaks, security findings, and hard-to-reproduce hangs comes from.

What fork copies, and what it shares

fork returns twice: once in the parent with the child’s pid, once in the child with zero. The child receives a copy of the descriptor table, not of the objects the table points at. Every entry in the child refers to the same open file description as the corresponding entry in the parent, which means a single shared offset, a single set of status flags, and a single reference count incremented by one.

That sharing is deliberate and useful. A shell that forks a command and redirects its output can rely on the child’s descriptor 1 referring to the same file the shell opened. A pipeline works because the two ends of a pipe are one object: the parent writes into one description, the child reads from the other, and the shared offset is not relevant because a pipe has no file position.

The same sharing creates the most common concurrency bug in this area. Two processes reading from a shared description advance one offset, so they split the input in an arbitrary interleaving rather than each getting a copy. A program that forks several workers to read different parts of a file must open the file separately in each, or use pread with explicit offsets, which does not consult or modify the shared position.

The child also inherits the parent’s signal dispositions, its session and process group, its resource limits, and its current working directory. File locks are the notable exception: fcntl-style record locks are not inherited by the child, while flock locks are held on the open file description and therefore are shared. A parent and child both holding a flock on the same description do not block each other, which is a different behaviour from two independent opens.

What exec does to the table

execve keeps descriptors open by default. POSIX requires this: a descriptor without the close-on-exec flag remains open across the call, referring to the same open file description, with the same offset and the same status flags. Only the per-entry flag FD_CLOEXEC causes an entry to be closed.

That default is the reason a long-running program leaks descriptors into every subprocess it starts. A server that holds a hundred client sockets, listens on a port, and keeps a log file open, then runs a helper program, hands all of that to the helper. The helper may not know the descriptors exist, but it can use them, and the leak has three distinct consequences.

The first is resource accounting: the helper holds references to objects it will never use, and the objects cannot be reclaimed while it runs. The second is a hang. If the helper is a filter reading standard input and the parent holds the write end of that pipe for another purpose, the helper never sees end-of-file, because the pipe still has a writer. The third is a security boundary violation: a descriptor to a privileged file, a listening socket, or a device node crosses into a program running with different privileges and different expectations.

Closing before exec, and the race that forces a flag

The obvious remedy is to close the unwanted descriptors in the child before calling execve. The loop is short: iterate from 3 to the limit and call close on each, or walk /proc/self/fd on Linux. Both are correct and both have a cost proportional to the descriptor limit, which for a process with a high RLIMIT_NOFILE can be tens of thousands of iterations of a system call on every spawn.

The loop also has a race in a multithreaded program. Between the fork and the execve, the child has exactly one thread, the one that called fork. But if that thread closes a descriptor by number and another thread in the parent had already duplicated a descriptor into the child’s number space, the close can hit the wrong entry. More importantly, open followed by fcntl(F_SETFD, FD_CLOEXEC) in the parent is not atomic: a concurrent fork in another thread can happen between the two calls, and the child inherits the descriptor without the flag.

The kernel provides the atomic forms to close that window: open with O_CLOEXEC, accept4 with SOCK_CLOEXEC, pipe2 with O_CLOEXEC, socket with SOCK_CLOEXEC, and fcntl with F_DUPFD_CLOEXEC. Each sets the flag as part of the same operation that creates the descriptor, so no observer can see the entry without it.

#include <fcntl.h>
#include <spawn.h>
#include <unistd.h>

/* Atomic creation with close-on-exec already set. */
int fd = open("state.db", O_RDWR | O_CREAT | O_CLOEXEC, 0640);

/* The child gets exactly the three descriptors it needs. */
posix_spawn_file_actions_t actions;
posix_spawn_file_actions_init(&actions);
posix_spawn_file_actions_adddup2(&actions, fd, STDIN_FILENO);
posix_spawn_file_actions_addclose(&actions, fd);

pid_t pid;
char *argv[] = { "worker", NULL };
posix_spawn(&pid, "/usr/bin/worker", &actions, NULL, argv, environ);
posix_spawn_file_actions_destroy(&actions);

posix_spawn is the portable answer to the whole problem. Its file actions are applied in the child after fork and before exec, in a defined order, so redirections and closes happen without a window in which another thread can interfere. Implementations on Linux use clone with CLONE_VFORK and CLONE_VM, which avoids copying the parent’s page tables at all — a meaningful saving for a large process. The interface also has limitations: it cannot express arbitrary work in the child, and the set of available actions is small.

The descriptors that must survive

Some descriptors are expected to cross. Standard input, output, and error are the obvious ones, and a shell sets them up deliberately before exec. Beyond those, a program may pass a descriptor to a helper by clearing FD_CLOEXEC on it just before the spawn, or by leaving it unset at creation.

Passing a descriptor this way requires care with the flag’s scope. The flag lives on the table entry, not on the description, so clearing it affects only the calling process’s entry. The child gets its own entry with the flag as it was at the moment of the fork, and the parent’s entry keeps whatever state it had. Two children spawned from the same parent see independent flags.

The value of a descriptor number is also not stable across exec. A program that relies on a descriptor being at a particular number must arrange that number explicitly, usually with dup2, which closes any existing entry at the target and installs the duplicate atomically. That is the mechanism behind the shell’s redirection syntax, and it is why descriptor numbers below 3 have a conventional meaning that programs rely on. The allocation rule that produces the lowest free number is examined in file descriptors as table indices.

What the child does with the number

A descriptor inherited across exec can be used by the new program in three ways, and only one of them is safe without agreement.

The first is to use it as an ordinary descriptor on a path the program knows. This is what happens with standard input and output and is fine because the convention is universal. The second is to treat the number as a private channel established by the parent, which requires the parent and child to agree on the number and on the object’s semantics. The third is to pass it onward, which is what a wrapper program does when it starts the real worker.

For the second and third cases the number should be fixed by dup2 rather than inherited at whatever slot happened to be free, and the object should be identified by something stronger than its number. On Linux, fstat on the descriptor yields the device and inode, which lets a program verify that the descriptor it received is the object it expects. Without a check, a child can receive a descriptor that is a regular file when it expected a socket, and the failure appears as a protocol error much later.

Leaks that outlive the spawn

The final wrinkle is that a descriptor inherited by a child keeps the object alive after the parent closes its own copy. A socket whose parent has finished with it stays half-open while a long-lived helper holds a duplicate, and the peer sees a connection that never closes. The reference count on the open file description is what governs this, and it is incremented by the fork and decremented by the close, in each process independently.

Diagnosing this requires knowing which process holds a reference. On Linux, /proc/<pid>/fd lists the entries for a running process and their targets, so a descriptor to a deleted file shows as the path with (deleted) appended, and a socket shows its inode. Walking the directory for each process of interest gives the answer directly. The same information is what makes the lifetime of an object visible when the process that created it is long gone, a question that also arises for unlinked files held open, and for descriptors passed over a socket as described in UNIX domain sockets and descriptor passing.