Interrupt Controllers from the 8259A to Message-Signalled Interrupts

A processor has one interrupt pin. A machine has dozens of devices that each want attention. Everything between those two facts is a controller, and the history of that controller is a history of removing shared state: shared wires, shared acknowledge cycles, shared vector assignments. Each generation solved the previous generation’s problem by adding a layer of indirection, and each layer is still visible in the kernel’s interrupt code.

The first controller was a pair of chips that could distinguish eight lines. The last is not a controller at all in the physical sense: a device writes a small value to a fixed memory address and the write itself is the interrupt.

Eight lines, then fifteen, then a cascade

The 8259A programmable interrupt controller had eight input lines, one output, and a set of programmable registers. A device pulled a line high; the controller compared the line against its mask register, resolved priority, and raised its output. The processor acknowledged, and the controller placed a vector on the data bus during the acknowledge cycle. The vector was a base value from the controller’s initialization plus the line number, which is why the pair was traditionally programmed to deliver vectors in the range from 0x20 to 0x2F.

Two chips could be cascaded: the slave’s output fed the master’s line 2, giving fifteen usable inputs and a known quirk. Line 2 of the master is occupied by the cascade, so a spurious vector 7 is reported when the slave raises its output and then withdraws it before the acknowledge cycle. The kernel counts these and continues; the handler for vector 7 is a stub with no device behind it.

The controller had real limitations that shaped driver code for years. Its priority order was fixed by line number, not by anything the software could configure, so a high-rate device on a low-priority line could be starved by chattering on a higher one; masking a line removes it from the rotation, but it cannot promote another. Its trigger mode was a single bit in the initialization command word that applied to all eight inputs at once, so a board mixing edge-triggered and level-triggered devices on one chip had to be wired to suit the majority. It had no notion of which CPU should receive an interrupt, because there was only one. And the acknowledge cycle was a bus transaction that had to be completed before the handler could run, which put the controller on the critical path of every interrupt.

The APIC pair

The advanced programmable interrupt controller split the problem in two. The local APIC sits in each processor, holds a vector table, a task priority register, and a timer, and delivers interrupts to its own core. The IOAPIC sits on the bus, holds a redirection table with one 64-bit entry per input line, and forwards an incoming assertion to a chosen local APIC by writing to the system bus.

The redirection entry is where the modern model becomes visible. It holds the vector number, the destination APIC identifier, the delivery mode, the trigger mode (edge or level), the polarity, and a mask bit. Because the vector is a field rather than a hardware consequence, the kernel can assign vectors to balance the table, avoid the exception range, and reassign a line when a device moves. Because the destination is a field, the kernel can steer a line to a particular CPU, which is the basis of interrupt affinity.

Delivery is a bus message rather than a dedicated wire. The IOAPIC writes to the local APIC’s address space; the local APIC decides whether the priority of the incoming vector exceeds its current task priority and, if so, raises the interrupt. This is also how inter-processor interrupts work: one core writes to another core’s local APIC to request a TLB shootdown or a reschedule. The mechanism is identical, which is why the kernel’s IPI code and its device interrupt code share entry paths.

The 8259A did not disappear immediately. Machines carried both, with the legacy pair routed through the IOAPIC’s inputs so that an operating system which did not understand the newer controller could still boot. The kernel’s support for the older controller survives in the code that masks it during startup, precisely so that the newer one has exclusive control.

Why wires stopped scaling

The IOAPIC solved priority and routing but kept the wire. Every device needed a dedicated input, and a machine with more devices than inputs needed either more controllers or shared lines. Sharing worked: PCI defines four interrupt lines, conventionally labelled INTA through INTD, and a slot’s lines are rotated by slot number so that adjacent slots do not all use the same one. A shared line meant every driver on it had to be called on every assertion, and each had to decide whether the interrupt was its own.

The cost is not just the extra call. A shared level-triggered line cannot be masked by one driver without silencing the others, so the flow handler masks the line globally and unmasks it after the action list completes, deferring that unmask to the end of a threaded handler when one was woken on an IRQF_ONESHOT line. A slow handler on the line delays every other device on it. And the rotation scheme was a workaround for having only four lines, not a design that scaled to a machine with thirty endpoints.

The interrupt becomes a write

Message-signalled interrupts remove the wire. A device with MSI capability is given an address and a data value during configuration; when it wants to interrupt, it performs a memory write to that address with that data. The write lands in the local APIC’s address range, and the low byte of the data is the vector.

/* The message fields of the MSI capability, which the kernel programs at enable time.
 * The capability's own header occupies the two bytes before message_address_low.
 */
struct msi_message_fields {
    uint32_t message_address_low;   /* 31:20 = APIC base address, 19:12 and 11:5 = destination id */
    uint32_t message_address_high;  /* destination id bits 8 and above; zero with xAPIC */
    uint16_t message_data;          /* bits 7:0 = vector, bits 10:8 = delivery mode */
    uint16_t reserved;
    uint32_t mask;                  /* per-vector mask bits, present only if the device supports it */
};

/* A single write by the device, at interrupt time, is the entire signal. */
*(volatile uint32_t *)msi_address = msi_data;

The consequences are structural. There is no acknowledge cycle, so the controller is no longer on the critical path. There is no shared line, so no driver has to disambiguate. And because the device writes a value the kernel chose, one function can own several vectors, which is the basis of MSI-X.

MSI-X extends the capability to a table of up to 2048 entries, each with its own address, data, and mask bit, held in a BAR rather than in configuration space. A network adapter with eight receive queues can request eight vectors and have each queue’s interrupt land on a different CPU, with no shared state between them. The mapping from queue to vector is decided at configuration time and stored in the device, so the data path is a single store.

The table size is the reason MSI-X exists as a separate capability rather than an extension of MSI. Configuration space is small, and 2048 entries of 16 bytes each — address, data, and a control word holding the mask bit — do not fit in it; a BAR-mapped table does. The mask bit per entry also allows a driver to silence one queue without touching the others, which the MSI capability cannot express: it grants a device a small number of vectors through a mask field sized in the capability, and carries no per-vector mask register of its own.

What the kernel does with the difference

The generic interrupt layer does not care which controller produced the vector, because the descriptor and chip abstraction described in From Hardware Line to Virtual IRQ Number presents the same verbs either way. What changes is the flow.

A level-triggered line needs mask-on-entry and unmask-on-exit, because the assertion persists until the device is serviced. An MSI is edge-like by construction: the write happens once, and there is nothing left asserted to re-trigger. The kernel therefore uses a flow handler for MSI that does not mask on entry, which removes two controller accesses from every interrupt. The saving is small per event and significant at high rates.

Affinity works differently too. A line’s destination is a field in the IOAPIC table that the kernel rewrites; an MSI vector’s destination is inside the address the device was given, so changing affinity requires reconfiguring the device. Both are exposed through the same sysfs interface, and both end up writing to hardware that the generic layer never sees.

That address is not necessarily used literally. Where interrupt remapping is enabled, the unit that owns the translation table rewrites the message — destination, vector, and delivery mode — before it reaches a local APIC, which is what lets the kernel restrict a device to a chosen set of vectors. The same table is the hook used when the destination CPU is running a guest: the translation can post the vector into the guest’s virtual APIC rather than deliver it to the host, the arrangement described in Interrupt Delivery Inside a Virtual Machine.

Failure modes at this layer

The first is a device that signals with the wrong data. Firmware may have programmed a message address that does not correspond to a valid vector, in which case the local APIC delivers a vector with no handler, the kernel logs an unexpected interrupt, and the device’s real interrupt is never serviced. The symptom is a device that works until its first interrupt, then stalls.

The second is the level-triggered line that is never de-asserted. Nothing in the controller can distinguish “device still asserting” from “driver forgot to clear”, so the interrupt repeats. Mask-on-entry bounds the damage to one wasted interrupt per handler completion instead of an unbounded storm.

The third is vector exhaustion. With MSI-X a driver may request more vectors than the platform can supply, and the kernel falls back to fewer, which changes the queue-to-CPU mapping the driver expected. Drivers are required to handle a smaller allocation than requested; those that do not fail in ways that look like lost interrupts under load.

The fourth belongs to the older controllers. A machine that falls back to the legacy pair because the IOAPIC was not described correctly in firmware will appear to work while silently losing interrupts under load, since the legacy controller has no way to report a lost edge. The kernel’s boot log records which controller was selected for exactly this reason.

Above this layer, the vector arrives at the entry stub and the descriptor’s flow handler runs the action list. When the work cannot be done there, it moves to the deferred mechanisms covered in Top Halves and Bottom Halves, and for network devices specifically to the polling scheme in NAPI and Interrupt Mitigation.