Operating Systems · Processes and Threads

What Threads Share and What They Own

A thread is a process that was created with fewer sharing flags, and the list of flags is the whole difference. What each shared resource costs.

The distinction between a process and a thread is presented in introductory material as a category difference: processes are isolated, threads share. In an implementation that provides both from one primitive, the difference is a set of bits. A new execution context is created by a system call that takes a mask of resources to share with the caller; passing almost nothing yields what the vocabulary calls a process, passing almost everything yields what it calls a thread, and the flags in between define the real design space.

One Primitive, a Mask of Shares

The Linux clone call takes a flags word and a pointer to a new stack. Each bit names a resource that the child will share rather than copy. CLONE_VM shares the address space: both tasks translate through the same page tables and see each other’s writes immediately. CLONE_FILES shares the file descriptor table, so a descriptor opened by one becomes visible to the other and a close in one is visible to both. CLONE_FS shares filesystem context, meaning the current working directory and the file creation mask. CLONE_SIGHAND shares the table of signal handlers, and sharing it requires CLONE_VM because handlers execute against the shared address space. CLONE_THREAD places the child in the same thread group, which makes it share the process ID for most purposes and appear as a separate thread identifier; since Linux 2.5.35 the flag may only be used together with CLONE_SIGHAND. CLONE_PARENT makes the child a sibling rather than a child of the caller, and a task created with CLONE_THREAD gets its parent this way whether or not the flag is given.

The absence of a flag means a copy. Without CLONE_VM the child gets a duplicate of the address space, which is where copy-on-write enters as described in What fork Copies, and Why Copy-on-Write Defers Almost All of It. Without CLONE_FILES the descriptor table is duplicated, and the duplicates refer to the same open file descriptions with their shared offsets. What is duplicated is the array of descriptor numbers; the O_NONBLOCK status flag lives in the open file description the two tables share, so setting it in one process affects both, while FD_CLOEXEC is a per-descriptor flag that each copy holds separately.

fork is this call with no sharing flags. Creating a POSIX thread is this call with the sharing flags that the C library selects. The two library functions hide a single kernel mechanism, and the intermediate combinations are legal and occasionally useful: a child that shares descriptors but not memory, or shares memory but not signal handlers, is expressible and behaves as the mask describes.

The Address Space Is the Real Boundary

CLONE_VM dominates every other flag in consequence. Tasks sharing an address space can read and write each other’s stacks and heaps with ordinary loads and stores, with no kernel involvement and no permission check. This is what makes thread communication cheap and thread bugs invisible to the memory protection hardware.

The consequence is that memory safety within a thread group is a language-level property, not a kernel-enforced one. A stray pointer in one thread can corrupt another thread’s stack, the allocator’s metadata, or a lock word. There is no fault and no diagnostic. Processes without CLONE_VM get the opposite trade: any attempt to touch the other’s memory faults, but every communication must go through a system call or a shared mapping.

CLONE_VM also means a thread cannot be given its own copy of a global variable. Storage duration and linkage are per-address-space, so a global declared static is one object seen by every thread. The per-thread instance must come from a different mechanism, thread-local storage, which is allocated per task and reached through a register or a per-task pointer rather than through a fixed address. The details of that allocation and its interaction with the stack are covered in Thread Stacks, Guard Pages, and Thread-Local Storage.

Sharing That Is Not Obvious

File descriptors are shared only with CLONE_FILES. With that flag, open in one thread can return a descriptor number that another thread is about to use, and close in one thread invalidates the number for all of them. A library that opens a temporary file and closes it can therefore close a number another thread has already reused, because from the kernel’s point of view the two operations are on the same table. POSIX requires each function in the standard to be thread-safe unless documented otherwise, and the standard resolution is that concurrent use of the same descriptor by two threads is the application’s responsibility; no library can make a shared descriptor number safe by itself.

Signal handlers are shared with CLONE_SIGHAND, but the mask of blocked signals is per task. This asymmetry is deliberate: a handler is code, and code lives in the address space, so sharing handlers follows from sharing memory. The decision to deliver a signal to a specific thread depends on which thread has it unblocked, and the process-directed delivery rules are the subject of Signal Delivery, Thread Masks, and Which Thread Runs the Handler.

The filesystem context (working directory, root directory, umask) is shared with CLONE_FS. A thread that calls chdir moves every other thread in the group, because there is one working directory per filesystem context, not per task. This is why chdir in a thread is nearly always a bug in a program that assumed process-wide state was thread-local.

Credentials are shared unconditionally on Linux, with no flag to opt out: the user and group IDs, and the capability sets, belong to a structure that all threads in a group reference. A thread that changes its effective user ID changes it for the group. Attributes that the kernel stores in the task structure rather than in the shared credential are per task, and the CPU affinity mask is the clearest example: one thread can be pinned to a processor without affecting its siblings, which makes affinity one of the few scheduling properties a thread holds privately.

The kernel also keeps per-task state that has no natural process-wide owner: the record of which locks a task holds, used to report a mutex that was left locked by a thread that died, and the alternate signal stack. The alternate stack is what lets a thread run a handler for a stack-overflow fault, so it must be established for each thread that needs one rather than assumed from the thread that created it.

Cost of a Second Execution Context

A thread’s kernel-side cost is a task descriptor plus a kernel stack. The kernel stack is a fixed allocation, sized to fit the deepest call chain the kernel can produce on that architecture; it is separate from the user stack and is not growable. The user stack, in contrast, is allocated by the thread library, with a size chosen by the application or by a default.

The user stack is the expensive part in practice. A default thread stack of a few megabytes multiplied by many threads consumes address space even when untouched, because the pages are allocated lazily and only committed on first touch. A program that creates thousands of threads with default stacks exhausts the address space or the commit limit long before it exhausts physical memory, and the fix is a smaller explicit stack size chosen from the actual depth the thread needs.

Switching between threads in the same address space avoids the page-table switch that a process switch requires. The processor’s translation cache and the branch predictor state remain valid, because the mappings did not change. This is the main reason thread-based servers can absorb frequent handoffs better than process-based ones, where every switch to a task in a different address space invalidates or must retag the translations. The cost is not zero: the register state must still be saved and restored, the kernel stack must change, and the scheduler’s bookkeeping runs identically. The mechanics of what is actually saved are in The Anatomy of a Context Switch.

The exit path differs too. When a thread returns from its start routine, it exits without destroying the address space, because other threads are still using it. Only when the last thread in the group exits does the process become a zombie and release its mappings. A thread that calls the process-exit system call terminates the whole group, which is why the C library distinguishes returning from the start function (thread exit) from calling exit (process exit).

Where the Model Leaks

Two details break the clean story. The first is that a shared address space does not make all state shared: the floating-point and vector register state, the signal mask, the alternate stack, the set of pending signals directed at that thread, and the thread-local storage block are per task even though memory is shared. The list is not derivable from the sharing flags; it is a property of the architecture and the kernel’s design.

The second is that CLONE_THREAD changes the meaning of process identifiers. Each task has its own identifier, the thread identifier that gettid returns; tasks in a group also share the group’s identifier, the thread group identifier, which is what getpid returns and which equals the thread identifier of the task that created the group. Signals sent to the group identifier are process-directed; signals sent to a thread identifier are thread-directed. Code that assumes a process identifier uniquely names a schedulable entity will be wrong about any multithreaded program.

This is why the same kernel object serves as both process and thread, and why the vocabulary keeps slipping: the implementation has one kind of thing, the task, and two names for configurations of it. The names describe intent and the flags describe behaviour, and where they disagree the flags are correct.

References

  1. clone(2) — create a child process — Linux man-pages project
  2. POSIX.1-2017, System Interfaces — pthread_create() — IEEE Std 1003.1-2017
  3. Programming with POSIX Threads — David R. Butenhof; Addison-Wesley
  4. Linux Kernel Development, 3rd edition — Chapter 3, Process Management — Robert Love; Addison-Wesley