Operating Systems · Interrupts and Timers

How a Hardware Interrupt Reaches a Handler on x86-64

The gate descriptor lookup, the privilege transition, and the stack switch that carry a device signal into kernel code, and what each step costs.

The processor knows nothing about network cards. It executes whatever instruction the program counter names, one after another, until a signal arrives on a pin that the interrupt controller has been programmed to treat as a request. At that moment the current instruction stream is suspended, a small amount of architectural state is written to memory, and control transfers to an address the processor looked up in a table. Every facility the kernel offers for devices begins at that transfer.

The interesting part is not that the jump happens but how narrowly it is specified. The processor does not consult the kernel, run a callback, or ask permission. It reads a descriptor, checks a privilege level, possibly switches stacks, pushes a frame, and jumps. The kernel’s freedom lies entirely in what it puts in the table beforehand.

One table, 256 entries

On x86-64 the table is the interrupt descriptor table. It holds 256 entries of 16 bytes each, so the largest possible table is 4096 bytes, and the IDTR register holds a base address and a 16-bit limit. The lidt instruction loads both; each logical CPU loads its own copy, which is why the table is described as per-CPU even though its contents are usually identical everywhere.

The entry format is not a plain function pointer. A 64-bit gate descriptor splits the target address across three fields, names a code segment selector, and carries a small set of attribute bits.

struct idt_gate {
    uint16_t offset_low;    /* bits 15:0 of the handler address   */
    uint16_t segment;       /* code segment selector, usually __KERNEL_CS */
    uint8_t  ist;           /* bits 2:0 select an IST stack, 0 = none */
    uint8_t  type_attr;     /* P | DPL | 0 | gate type            */
    uint16_t offset_mid;    /* bits 31:16                         */
    uint32_t offset_high;   /* bits 63:32                         */
    uint32_t reserved;      /* must be zero                       */
};

The type_attr byte packs four things: a present bit, a two-bit descriptor privilege level in bits 6:5, a bit that must be zero, and a four-bit gate type in bits 3:0. Type 0xE is a 64-bit interrupt gate; type 0xF is a 64-bit trap gate. Those four bits sit at the bottom of a packed 16-bit field, which is why a gate whose ist field is zero carries exactly 0x8E00 in the bytes that hold the interrupt stack index, the type, the privilege level, and the present bit together. The ist field is three bits wide and selects one of the seven interrupt stack table entries held in the task state segment. A value of zero means the handler runs on whatever stack the interrupted context was using.

The table is not writable once the kernel is running. The page holding it is mapped read-only immediately after the gate set is installed, and installation is gated on a flag that debug builds check, so a later attempt to add a vector after boot is a warning rather than a silent change. That is the practical form of the claim that the table is effectively immutable. On 32-bit builds the descriptor is half the size, since the high half of the target address does not exist.

Vector assignment is fixed at the low end. Vectors 0 through 31 belong to exceptions the processor raises itself: vector 0 for divide error, 1 for debug, 2 for the non-maskable interrupt, 3 for breakpoint, 6 for invalid opcode, 8 for double fault, 13 for general protection, 14 for page fault. Vectors 32 through 255 are available for external hardware and for software interrupts, and the boundary matters because the interrupt controller can only deliver in the upper range.

Interrupt gates, trap gates, and one flag

An interrupt gate and a trap gate differ in a single behaviour: the interrupt gate clears the interrupt flag in RFLAGS as part of the transition, so maskable interrupts stay blocked until the handler executes sti. A trap gate leaves the flag alone, so a higher-priority interrupt can arrive while the handler is still running.

The choice is a re-entrancy decision, not a performance one. Handlers that manipulate state which a nested interrupt would also touch are installed behind interrupt gates. Exceptions are almost all interrupt gates for this reason, including the page fault handler, whose per-CPU state cannot survive re-entry from an arbitrary interrupt. Trap gates are used where re-entry is intended, most conspicuously the overflow exception, which has to reach a signal handler while the flags of the interrupted context are preserved. Giving up the cleared flag elsewhere would cost a needless sti/cli pair on every occurrence, but that is a secondary consideration next to correctness.

The installation path makes the convention visible. The kernel’s exception vectors are declared in a table whose entries name the gate type explicitly: the divide-error, invalid-opcode, general-protection, and page-fault entries are interrupt gates at privilege level 0, the debug and breakpoint entries are interrupt gates reachable from user space, and the entries that need their own stack are interrupt gates that additionally name an interrupt stack table index. The overflow exception is the one declared as a trap gate. The table is also where the gate types for the local APIC timer, the thermal interrupt, and the error-reporting banks are set, before the generic loop fills in the remaining external vectors with the same interrupt-gate form.

The flag is also the reason cli and sti appear in driver code at all. Disabling interrupts on the local CPU does not stop other CPUs from running the same handler, and it does not stop a non-maskable interrupt. It only prevents the local processor from taking another maskable interrupt, which is precisely what a short critical section against a handler on the same CPU needs.

The stack switch

When the target code segment has a higher privilege than the interrupted one, the processor cannot push the frame onto the current stack, because that stack belongs to the outer privilege level and is not trusted with kernel state. It reads the ring-0 stack pointer from the task state segment, switches RSP, and then pushes, in order, the old stack segment (padded to eight bytes), the old stack pointer, RFLAGS, the old code segment (likewise padded), and the instruction pointer. That is 40 bytes. If the privilege level did not change, the segment registers are already correct and only 24 bytes are pushed. Exceptions that carry an error code push it as well, after the instruction pointer, so a page fault against a fresh ring-3 frame occupies 48 bytes.

If the gate’s ist field is nonzero, the processor takes RSP from the corresponding interrupt stack table slot instead. The old SS and RSP are pushed even though the privilege level may not have changed, which keeps the frame layout uniform. Linux gives each CPU a dedicated 16 KiB interrupt stack, four pages, and routes external interrupts onto it. The point is not speed but survival: a handler that runs on the task’s kernel stack inherits however little of that stack remains after a deep call chain, and a device interrupt has no relationship to the depth of the call it interrupted.

Double fault, non-maskable interrupt, and machine check have their own stack table entries. A double fault is raised precisely when the processor cannot deliver an earlier exception, most often because the stack for that exception was unusable, so it must run somewhere else by construction. If delivering a double fault also fails and no further stack switch is available, the processor raises a triple fault, which is not an exception at all: the machine resets.

Where the vector comes from

An external device does not present a vector. It asserts a line, and the interrupt controller decides whether that line is masked and which vector it corresponds to. On a machine with an IOAPIC, each redirection table entry holds the vector, the destination CPU, the trigger mode, and a mask bit; the controller delivers the vector to a local APIC, which interrupts its own processor. With message-signalled interrupts, the device performs a memory write to a fixed address region and the vector is encoded in the low byte of the data, so there is no line to share and no redirection table entry at all.

Because the vector is programmed rather than wired, the mapping from device to vector is kernel policy. Linux keeps it in an interrupt domain, and the number the driver receives is a virtual interrupt number rather than the hardware one; the indirection is described in From Hardware Line to Virtual IRQ Number.

Entry code and the action chain

After the jump, a small assembly stub saves the general-purpose registers the interrupted code was using, builds a pt_regs frame, and calls into the generic interrupt layer. That layer converts the vector back into a descriptor, selects a flow handler based on the interrupt type, and the flow handler walks a linked list of actions registered on the line. The list exists because lines can be shared: IRQF_SHARED allows several drivers on one line, and every handler on it is called and expected to report whether its own device was the source.

Between irq_enter and irq_exit the CPU is in interrupt context, and the kernel tracks that with a per-task preemption count rather than a global flag. The distinction is what makes sleeping in a handler a detectable bug instead of an intermittent corruption. Anything that might sleep, including a mutex, a page allocation that can reclaim, or a copy to user space, is illegal there; the constraints and their consequences are the subject of Top Halves and Bottom Halves.

What the transition costs

Entry and exit together cost on the order of hundreds of cycles: the table read, the privilege transition, five pushes, the segment reloads, the register save, the descriptor lookup, the action chain, and the symmetric restore. The input to that number is a count of the memory accesses along the path, not a timing run; the actual figure depends on the microarchitecture and on cache state, and only its order of magnitude matters for design. The arithmetic it feeds needs no instrumentation. A device interrupting a hundred thousand times per second leaves tens of millions of cycles per second in entry and exit, which is a few percent of a modern core; a million interrupts per second claims a core outright. This is the reason high-rate devices are pushed toward polling or interrupt coalescing, a trade examined in NAPI and Interrupt Mitigation.

Where the mechanism breaks

The first failure is the unacknowledged level-triggered interrupt. If the handler returns without telling the controller that the device has been serviced, the line stays asserted, the controller re-presents the same vector immediately, and the machine spends all of its time entering the same handler. The generic layer masks a level-triggered line on entry and unmasks it on exit partly to make this survivable: the interrupt is not lost, but it cannot recur until the handler finishes.

The second is re-entrancy through an early sti. A handler that enables interrupts before it has finished with shared per-CPU state can be interrupted by a handler that touches the same state. The interrupt gate’s cleared flag is the default protection, and giving it up requires the same reasoning as any other lock.

The third is the spurious interrupt. The legacy 8259A pair can deliver vector 7, or vector 15 through the slave, with no device behind it; the kernel counts the event and returns. Modern controllers make this rarer but not impossible, because a device can withdraw its request between the acknowledge cycle and the handler.

The fourth is a stale table. Since each CPU loads its own IDTR, a change to a gate on one processor is not visible to the others until they reload, and a CPU that is offline or spinning in a low-power state may hold a descriptor set that no longer matches the kernel’s intent. The kernel’s interrupt entry path treats the table as effectively immutable after boot for this reason.

What follows in this cluster is a consequence of the constraints established here. The handler cannot sleep and must be short, so work moves downward into softirqs and kernel threads. The vector is programmable, so it needs an indirection layer. And the timer that generates periodic interrupts is itself a device that arrives through the same gate, which is why clock handling and interrupt handling share so much machinery.

References

  1. Intel 64 and IA-32 Architectures Software Developer Manual, Volume 3A: System Programming Guide — Intel Corporation
  2. AMD64 Architecture Programmer Manual, Volume 2: System Programming — Advanced Micro Devices
  3. Generic IRQ Handling — The Linux kernel documentation