Open File Descriptions and the Offsets They Share

Two processes open the same file for reading, each reads a hundred bytes, and each receives the first hundred bytes of the file. One process opens the file, forks, and both parent and child read a hundred bytes: together they consume two hundred bytes, and neither sees what the other read. Both statements describe correct behaviour of the same interface, and the difference is not in the file or in the inode. It is in how many intermediate objects were created between the descriptor and the file.

Three layers, three lifetimes

A descriptor is a small integer indexing a per-process table. The entry in that table points at an open file description. The description points at an inode. Each layer has a different owner and a different lifetime, and each answers a different set of questions.

The descriptor table belongs to the process. It is what fork copies and what exec preserves unless a close-on-exec flag is set, and it is what a program closes to release its reference. Descriptor numbers are allocated lowest-first, which is why 0, 1, and 2 are conventionally standard input, output, and error, and why a program that closes its standard output and then opens a file finds the file on descriptor 1.

The open file description belongs to the kernel and is shared by every descriptor that refers to it. It holds the current offset, the access mode the file was opened with, the status flags (O_APPEND, O_NONBLOCK, O_SYNC), the file’s credential snapshot where relevant, and the reference to the inode. It has no name and no per-process state.

The inode belongs to the filesystem. It holds the data location, the size, the permission bits, and the link count, and it is shared by every description open on that file.

Operation	Descriptor table	Open file description	Inode
open	new entry	new	shared
dup, dup2, fcntl(F_DUPFD)	new entry	shared	shared
fork	copied	shared	shared
close	entry removed	reference dropped	reference dropped
read, write	unchanged	offset advanced	data touched
lseek	unchanged	offset set	unchanged

The table is the whole explanation of the two opening scenarios. Two separate open calls produce two descriptions, each with its own offset starting at zero, so both processes read the first hundred bytes. fork duplicates the descriptor table but not the descriptions, so parent and child share one offset and their reads advance it together.

What sharing costs and what it buys

Shared offsets make a common idiom work: a parent process opens a log file, forks a worker, and the worker writes through the inherited descriptor with O_APPEND set, so that writes from both processes append at the current end rather than at a remembered position. Without O_APPEND, a shared offset means writes land consecutively from wherever the offset happens to be, which is correct for a pipeline of cooperating processes splitting one stream and wrong for independent log records.

Shared offsets also make lseek a hazard. A program that positions a shared descriptor affects every process holding it. This is the reason libraries that wrap a descriptor in buffered I/O and then fork produce duplicated output: the buffered data in the parent’s FILE object is flushed by both processes on exit, and if the underlying offset is shared the second flush writes at a position the first flush already passed. The standard’s guidance to flush before fork exists because the buffer, not the kernel, is the part that gets copied.

Independent offsets, by contrast, make concurrent readers safe. Two processes reading different regions of a large file, each with its own descriptor, cannot interfere, and no locking is required because no state is shared. That is why parallel index builds and pread-based readers open the file once per thread.

pread and pwrite exist because the alternative of lseek followed by read is not atomic with respect to other processes sharing the description. The pair of calls allows another process to move the offset in between. pread takes the offset as an argument and does not change the description’s offset at all, so it is safe on a shared description and leaves the file position untouched. A reader that wants a stable position across a shared descriptor must use pread, not lseek.

The offset is a property of the description, not the file

Nothing about the offset is stored on disk. A file has no persistent read position; a program that closes and reopens starts at zero, and a program that crashes leaves no position behind. This is why tail -f must remember where it stopped and seek there on each reopen, and why it cannot simply ask the file where reading left off.

The offset is also a single number even when the description is shared by readers at different logical positions. Two processes that want independent positions must hold independent descriptions, and the only way to obtain one is another open. There is no interface for cloning a description without cloning its offset, except that open of the same path produces a fresh description by construction.

Append mode is the exception that makes shared offsets useful. With O_APPEND, each write sets the offset to the current end of file before writing, as part of the same operation. The check and the write are not separable by another process, so two writers appending to one file through separate descriptions do not overwrite each other’s data. Without the flag, the same two writers race: both read the size, both compute the same offset, and one write overwrites the other. The race window is small but the interface provides no way to close it from user space other than O_APPEND or an explicit lock.

O_APPEND does not make the offset irrelevant. Reads still use the description’s offset, and after a write in append mode the offset is left at the end of the written data, so a subsequent read on a read-write append descriptor reads from there rather than from the start.

Flags that live in two different places

Not every flag passed to open lands in the same object, and the split matters for fcntl.

Access mode (O_RDONLY, O_WRONLY, O_RDWR) and the status flags (O_APPEND, O_NONBLOCK, O_ASYNC, O_DIRECT) are properties of the open file description. They are set at open and changed afterwards with fcntl(F_SETFL), and because the description is shared, a change affects every descriptor referring to it. Setting O_NONBLOCK through one descriptor makes reads through a duplicated descriptor non-blocking as well.

Descriptor flags are different. FD_CLOEXEC is stored in the descriptor table entry, not the description, and fcntl(F_SETFD) changes it for one descriptor only. A process that dups a close-on-exec descriptor gets a new descriptor without the flag unless it sets it again. That asymmetry is why O_CLOEXEC at open time is preferable to setting the flag afterwards: the window between open and fcntl is a window in which a concurrent fork in another thread produces a child holding an unintended descriptor.

The access mode cannot be changed at all after open. There is no fcntl command to turn a read-only description into a read-write one, because the permission check happened at open time and the description records its result. A program that needs both must close and reopen, and it must accept that the inode may have changed in between.

Reference counting and the end of a file

A description is destroyed when the last descriptor referring to it is closed. Until then, the description keeps the inode alive even if every name for the file has been removed. This is the mechanism behind reading a file that has been unlinked, described in Unlink While Open and the Life of an Orphaned Inode, and it is also why a long-running process can pin disk space for a file that no longer appears in any directory.

Closing a descriptor is not the same as flushing data to storage. close drops a reference and, when the last reference goes, the kernel may still hold dirty pages for the file. Durability requires an explicit call, covered in fsync, fdatasync, and What Survives a Power Loss.

fork interacts with all of this in one further way: the child inherits the descriptor table, so the child’s descriptors refer to the same descriptions and the same inodes. A child that never closes them keeps the file open after the parent exits, and a parent that waits for the child to finish does not thereby release the description. Programs that pass descriptors to children must close them deliberately, which is what FD_CLOEXEC automates for the exec case.

The inode’s identity, by contrast, is independent of any description, and the pair of fields that names it is described in What an Inode Stores, and Why It Stores No Name. Confusing the two layers produces the classic bugs: assuming dup gives an independent position, assuming fork gives an independent position, or assuming that lseek on one descriptor cannot affect another process reading the same file through an inherited one.

Asking a descriptor what it refers to

The description’s reference to an inode is the authoritative answer to “what file is this process holding”, and it can be retrieved without a pathname. fstat fills the same structure from the description’s inode, which means a process can compare the file it opened with a file named later, and can do so after the name has been replaced. The comparison that matters is of the device number and inode number together; the descriptor’s offset and flags take no part in it.

The same query answers whether the name still points anywhere. A file whose link count reaches zero while a descriptor is open reports st_nlink of zero, and the data remains readable through the descriptor. That value is the only portable signal that the file has been unlinked, and it is available on every descriptor, not just on ones the program opened deliberately.

/proc/self/fd exposes the second half of the relationship: for each descriptor it presents a symbolic link whose target is a path to the file, or an annotation for objects that have no name in the filesystem. The path is reconstructed by the kernel from the inode, so it can name a different hard link than the one the program used, and it can be marked as deleted once the last name is gone. Tools that pass descriptors between processes rely on this representation to reopen an object by number, which is why the link is worth knowing about even though the portable interface never mentions it.

The practical consequence is that a descriptor is a stronger reference than a path. A path is resolved once, and whatever it named at that moment is what the description captured; every later operation through the descriptor acts on that object, while every later operation on the path resolves again and may find something else. Programs that need the object they checked rather than the name they checked open it once and then operate through the descriptor, and the resolution rules that make the difference visible are the subject of Path Resolution Component by Component.