Operating Systems · Boot and Firmware

From the First Kernel Thread to PID 1

How the kernel creates its initial task, what changes when that task returns to user mode, and why the first user-space process must be a single-threaded, privileged program that never exits.

The kernel reaches the end of its initialization with one thread running in kernel mode, a scheduler that has nothing else to run, and a filesystem that is mounted but not yet usable by anything. Turning that state into a system with processes is a specific sequence of steps, and the sequence exists because the first process cannot be created the way every later process is.

A normal process is created by an existing process calling fork or its equivalent. It inherits a parent’s address space, credentials, file descriptors, and signal dispositions, and the kernel can copy all of that because there is a parent to copy from. The first process has no parent. Its address space does not exist yet, its credentials have no source, and there is no user-space code to return to. The kernel constructs it directly, out of the kernel’s own data structures.

The initial task

On a kernel that uses the conventional Unix structure, the boot thread is a kernel thread: it has a task structure, a kernel stack, and no user-space memory map. It is often given a low identifier that is not PID 1, because PID 1 is reserved for the process that will run in user mode. The boot thread’s job is to perform initialization and then, in effect, become something else.

The transition happens at the end of start_kernel, the architecture-independent initialization function. Its last act is not to return, because there is nothing to return to. It calls a function that creates a kernel thread which will run the rest of the initialization in a context that can be scheduled, and then the original thread enters the idle loop: it waits for work, and when there is none, it executes the architecture’s halt instruction. From that point on, the machine is never without a runnable task, but the task that is running is the one that will be interrupted by everything.

Creating that first kernel thread uses the same mechanism as any later thread creation: the kernel allocates a task structure and a stack, initializes the register state to point at a function, and puts the task on a run queue. The register state is synthetic. It was never saved from a real context switch, so the kernel must fill it in by hand, including the flags word that the context-switch code will restore. A mistake here produces a task that faults on its first instruction with no parent to report it to.

The first user-mode entry

The kernel thread that runs the rest of initialization eventually calls a function whose name in the Unix tradition reflects what it does: it executes a program in user space, replacing the kernel thread’s context. The mechanism is a execve-like path taken from inside the kernel, with a pathname that is a constant: /sbin/init, or a value from the kernel command line, or a series of fallbacks tried in order.

What makes this different from a normal execve is where the credentials come from. The kernel thread has no user identity; it is running with full privileges and no user namespace membership that a user-space program would recognize. The kernel assigns the new process a root identity, gives it full capabilities, and marks it as the initial process in its namespace. Every later process’s credentials are derived, directly or indirectly, from this one.

The address space is created from scratch. The kernel opens the executable, reads its headers, maps its segments, builds a stack with the argument vector and environment, and sets the instruction pointer to the program’s entry. Then it returns to user mode. That return is the first transition from kernel to user in the system’s life, and the code path that performs it is shared with every later execve, which is why it is well tested.

The process that results is PID 1. It is single-threaded by construction, because the kernel created exactly one thread. It has no parent in the ordinary sense: the kernel records its parent as the idle task or as itself, depending on the design, and it is reparented to nothing. If it exits, the kernel treats that as a fatal condition and halts or panics, because there is no other process to adopt the orphans and no supervisor to restart it.

Why PID 1 has unusual privileges and unusual restrictions

The initial process runs as root with full capabilities, but the kernel applies signal restrictions that no other process has. Signals whose default action is to terminate are not delivered to PID 1 unless it has installed a handler, because the kernel assumes that a process which cannot be killed is the correct behaviour for the system’s supervisor. A SIGKILL sent to PID 1 by another process is discarded. The same restriction applies to signals generated by the kernel for resource conditions that would normally terminate a process, such as a fault in a memory mapping that cannot be resolved.

The reason is that PID 1 is the reaper. Every orphaned process is reparented to it, and it is responsible for calling wait on them so that their process-table entries are released. A system whose PID 1 has died has no reaper, so zombie entries accumulate and the process table fills. The kernel’s restriction is a crude but effective way of preventing the supervisor from being killed by an ordinary signal before it has arranged a replacement.

The restriction is also a source of confusion, because a program that runs as PID 1 in a container behaves differently from the same program run as an ordinary process. A shell script that relies on SIGTERM terminating it will not terminate when it is PID 1, and the resulting behaviour looks like a hang. Container runtimes address this by running a small init process as PID 1 that forwards signals and reaps children, with the application as a child.

What PID 1 does before anything else

The first user-space program’s job is to bring up everything the kernel deliberately did not. The kernel has mounted a root filesystem, but it has not mounted the other filesystems listed in the mount configuration; it has created the device nodes it needs, but not the ones the rest of the system expects; it has started no daemons, configured no network interface, and set no hostname. Those are policy decisions, and the kernel leaves policy to user space.

The sequence in a traditional system is a small set of steps in a fixed order: mount the remaining filesystems, start the service manager or run the init scripts for the configured runlevel, and then, once the system is up, remain running as the reaper. A modern service manager does the same work with dependency ordering: units declare what they require, and the manager starts them in an order that satisfies those declarations, in parallel where the graph allows.

The ordering constraint that matters most is the one between the root filesystem and everything else. The kernel mounts the root from either a block device it was told about or an initramfs. If it is an initramfs, the transition to the real root happens in user space, which means PID 1 is running from a temporary filesystem and must move itself to the real one. That handoff is the subject of unpacking an initramfs and switching to the real root.

Failure modes

If the executable named on the kernel command line does not exist, the kernel tries the next fallback. If none exists, it panics with a message naming the paths it tried. That message is the most common diagnostic for a system whose root filesystem was mounted but whose init program was not installed, which happens after a botched upgrade.

If the executable exists but faults immediately, the kernel’s fault handler prints the fault and the process is killed. Because it is PID 1, the kernel then panics rather than continuing without a reaper. The panic message includes the fault address and the instruction pointer, which is enough to identify a missing shared library only if the dynamic linker’s error output reached the console before the fault; if the fault is in the dynamic linker itself, the message is opaque.

A third failure is a PID 1 that starts successfully and then blocks forever on a step that cannot complete, such as mounting a filesystem whose device has not appeared. The kernel is healthy, the scheduler is running, and the system is unusable, with the only diagnostic being whatever the init program printed before it blocked. This is why init programs print progress and why boot-time timeouts exist.

The kernel thread that becomes PID 1 is also the thread that performed the last stage of initialization, and the state it leaves behind constrains what user space can assume. The memory map it built from the firmware’s description is still the map the page allocator uses, and the reservations recorded during that stage are still reserved. Those reservations are described in how firmware describes physical memory.

References

  1. Linux Kernel Development, 3rd edition — Robert Love, Addison-Wesley, 2010
  2. Advanced Programming in the UNIX Environment, 3rd edition — Stevens and Rago, Addison-Wesley, 2013
  3. POSIX.1-2017: Base Definitions and Headers — IEEE and The Open Group