The Three-Level Indirection Behind a Unix File Descriptor

The read call takes an integer. That integer is not an address, not a handle to a kernel object, and not a name. It is an index into an array owned by the process, and the entry it selects points at a second structure shared with other descriptors, which in turn points at the file itself. Three levels, each with a different lifetime and a different sharing rule. Almost every surprising behaviour of the interface follows from which level an operation copies.

What the integer indexes

Each process has a table of open descriptors. The table is per-process and private: a descriptor number means something only inside the process that owns it, and two unrelated processes holding descriptor 3 have no relationship unless one passed the descriptor through a socket or inherited it. The kernel allocates the lowest-numbered free slot, a rule that POSIX states explicitly for open and which makes descriptor numbers reusable and predictable rather than arbitrary. Close a descriptor and the next open in that process returns the same number.

Descriptors 0, 1 and 2 are conventionally standard input, standard output and standard error, established by the shell before it executes a program. The convention is not enforced by the kernel, only by the shell and by login; a process that closes descriptor 0 and opens a file will find that file on descriptor 0. The first three slots being spoken for is why a simple program that opens one file typically sees descriptor 3.

The table has a fixed size set by a resource limit, historically RLIMIT_NOFILE and adjusted by the shell with ulimit -n. The kernel’s traditional default soft limit is 1024 descriptors, with a hard limit considerably higher, though distributions and service managers commonly raise both; a process that opens beyond its limit receives EMFILE rather than a kernel-wide failure, and the system-wide table can also fill, reported as ENFILE. The 1024 default is old enough to shape APIs: select represents descriptor sets as bitmaps indexed by descriptor number, and FD_SETSIZE is 1024 in the common C libraries, so a descriptor above that bound cannot be used with select at all. poll and epoll were designed without the dense bitmap and therefore without the ceiling.

The open file description holds the state that moves

The table entry points at an open file description, sometimes called the file object or the open file structure. This is where the read offset lives, along with the status flags (O_APPEND, O_NONBLOCK) and the access mode the file was opened with. The description is not per-process. Several descriptors, in one process or across processes, can point at the same description, and they then share one offset and one set of status flags.

That sharing is the entire content of fork with respect to files. The child receives a copy of the descriptor table, but each entry in the copy refers to the same description as the parent’s entry. Parent and child therefore share an offset: if the parent reads 100 bytes, the child’s next read begins at byte 100, not at byte 0. dup behaves identically, producing a second descriptor for the same description with the same offset, which is why dup2 is the standard way to redirect a stream in place without losing the position.

Two independent open calls on the same path produce two descriptions and therefore two independent offsets, even though both point at the same inode. Sharing the file is not the same as sharing the position, and conflating the two is the mistake the model most readily invites. A program that opens a log file twice, once for reading and once for writing, has two offsets; the writer’s position is not the reader’s, so the writer can clobber data the reader has not yet consumed.

O_APPEND is a property of the description, not of the call, and it changes what a write means. With the flag set, every write atomically positions at the current end of file before transferring bytes, so two processes appending to the same file through separate descriptions do not overwrite each other’s records. Without it, the position is wherever the shared offset happens to be. This is the reason append-only logging works across processes without locking.

#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>

int main(void) {
    int a = open("data.txt", O_RDONLY);
    int b = dup(a);              /* same description: offset is shared */
    int c = open("data.txt", O_RDONLY);  /* new description: own offset */

    char buf[4];
    read(a, buf, 4);             /* a moves to byte 4 */
    printf("b reads at %ld\n", (long)lseek(b, 0, SEEK_CUR));  /* 4 */
    printf("c reads at %ld\n", (long)lseek(c, 0, SEEK_CUR));  /* 0 */
    close(a);
    close(b);
    close(c);
    return 0;
}

The block prints 4 and 0. Closing a does not invalidate b, because the description has a reference count and survives until the last descriptor referring to it is closed.

The inode is the third level, and the only one that outlives the process

Below the description sits the inode, the on-disk or in-memory record of the file: ownership, permission bits, size, timestamps, block pointers, and a link count. Everything that is true of the file rather than of this particular use of it lives here. The description adds the offset and the flags; the inode holds the identity.

The separation explains the semantics of unlink. Removing a name decrements the inode’s link count but does not free the inode or its data while any description still refers to it. A process that opens a file, unlinks it, and keeps reading sees a consistent file that no longer has a name. Only when the last descriptor closes does the count reach zero and the blocks return to the allocator. This is the mechanism behind temporary files that vanish on close without a cleanup step, and behind the behaviour where a deleted file’s space is not reclaimed until the process holding it exits.

It also explains why permissions are checked once, at open. The inode’s mode is consulted when the description is created; subsequent reads and writes are not re-authorised against the mode. A process that keeps a descriptor open across a chmod retains its original access, which is correct under the model but regularly surprises people who expect permission changes to revoke existing access.

Which call copies which level

The three levels map cleanly onto the operations that duplicate them.

Operation	Descriptor table	Open file description	Inode
open	new entry	new description	existing inode
dup, dup2	new entry	shared	same
fork	copied table	shared	same
exec	table preserved except close-on-exec entries	preserved	same
close	entry cleared	reference dropped	reference dropped

Reading the table down a column answers most questions about a program’s behaviour. After fork, the parent and child share offsets but not descriptor numbers. After exec, descriptor numbers are preserved, offsets are preserved, and the new program inherits whatever the old one left open. That last row is the reason close-on-exec exists as a flag rather than as a cleanup convention, and the race it closes is covered in the descriptor inheritance article.

Where the model breaks down

The offset is per description and not per descriptor, which means two threads reading the same descriptor in one process contend for one position. Each read advances it atomically, so bytes are not duplicated, but the split between them is arbitrary and the order of records is lost. pread and pwrite take an explicit offset and leave the description’s position untouched, and they are the only sane way to do positional I/O from multiple threads.

Record locking has a similar mismatch. POSIX fcntl locks belong to the process, not to the descriptor or the description, and closing any descriptor for the file in that process drops all the process’s locks on it. A library that opens, locks, and closes a file can therefore silently release a lock its caller believed it held. BSD flock locks attach to the description instead, so duplicated descriptors share a lock and closing one does not release it while another remains. The two interfaces coexist with incompatible semantics, a consequence of layering locking onto a structure that was not designed to carry it.

The final limitation is that the description is not visible. There is no portable way to ask how many descriptors refer to one description, or to enumerate the descriptors a process holds; /proc exposes that on Linux, as described in the procfs article, but it is an inspection interface rather than part of the model. Programs that need to reason about sharing must track it themselves, which is why passing descriptors between processes is usually wrapped in a protocol that also passes the intended semantics.