Masking Interrupts, Priorities, and Exclusive Access Without an OS

On a system with no kernel, there is no lock object to take and no scheduler to appeal to. The only concurrency is between the main flow of control and interrupt handlers, the only pre-emption point is an exception the software did not choose, and the whole set of tools for making a shared update atomic consists of three mechanisms: masking exceptions, excluding accesses at the processor level, and ordering compiler output so that the mechanisms are not undone after the fact.

The mask, and the nesting problem it creates

The simplest mechanism is a bit in a processor status register that stops configurable-priority exceptions from being recognised. Instructions exist to set and clear it in a single cycle, and to read it. Nothing about the main flow of control can be interrupted while the bit is set except the two exceptions the architecture never masks: the non-maskable interrupt and the fault handler that catches escalated faults.

The natural wrapper is a pair of calls, one to disable and one to enable. It is wrong, and it is wrong in a way that appears only under nesting. If a function that disables interrupts is called from the main loop and, while the mask is set, an unmasked fault occurs whose handler calls a function that re-enables interrupts, the mask is cleared on return to the main loop’s critical section, which then completes without protection. The correct pattern saves the previous state and restores it, so nesting composes:

static inline uint32_t enter_critical(void)
{
    uint32_t previous;
    __asm volatile ("mrs %0, primask\n\t"
                    "cpsid i"
                    : "=r" (previous)   /* PRIMASK as it was */
                    :
                    : "memory");        /* do not move accesses across this */
    return previous;
}

static inline void exit_critical(uint32_t previous)
{
    __asm volatile ("msr primask, %0" : : "r" (previous) : "memory");
}

The memory clobber is not decoration. The compiler has no model of interrupts: from its perspective the shared variable is only ever written by the code it can see, so it may keep the value in a register across the critical section or sink a store past the second call. The clobber forces memory state to be re-established at both ends. The second requirement is that the variable itself be volatile or an atomic type, for the reasons given in volatile and memory-mapped register access; masking and the clobber together do not make a non-volatile variable shared with a handler behave.

What the mask costs

The cost of disabling interrupts is not the cycles spent in the critical section. It is that every interrupt that could have been taken during the section is delayed until the section ends, so the worst-case interrupt latency of the system becomes the longest masked interval in the program plus the handler’s own execution time. A critical section that runs for a few cycles around a word update is nearly free. A critical section that spans a peripheral access, a loop, or a call into a driver adds its whole duration to the latency budget of every interrupt in the system, including ones whose deadlines are unrelated.

The second cost is visibility. A program’s interrupt latency is a property of no single function: it is the maximum over all paths of the time spent with exceptions masked. Tooling cannot compute it from the source, so it is maintained by convention, which is why masking belongs in short inline helpers rather than in driver entry points.

Priority masking as a narrower tool

Where the architecture provides a threshold register, a program can mask a subset of interrupts rather than all of them. The threshold is compared against the priority of each pending exception, and exceptions more urgent than the threshold continue to be taken. That makes the mask a priority filter: a section that shares data only with low-priority handlers can set the threshold at their priority level, which masks them and everything less urgent and leaves a more urgent handler free to run.

Two properties make this harder to use than it looks. The priority field on the common implementations is byte-wide while only the most significant bits are implemented, so the value written to the threshold must be shifted left by the number of unimplemented bits or it names the wrong priority entirely. And masking at a threshold never protects against an interrupt whose priority is above the threshold: if the shared data is also touched by a handler at higher priority, the mask is a no-op with respect to the code that actually races. Deciding which mechanism applies requires knowing the priorities of the handlers that touch the data, not just the data.

Exclusive accesses and retry loops

The processor-level alternative is a pair of instructions: an exclusive load that also arms a monitor, and an exclusive store that succeeds only if the monitor is still armed. If an exception intervenes and returns, the architecture clears the local monitor, so the store fails, and a compare-and-swap loop retries. The value of this is that the critical section is implicit and its duration is the retry loop, so the worst-case masked interval — and therefore the interrupt latency — stays at zero.

The cost is that a retry loop needs a contended-free assumption to terminate. On a single core with one interrupt source the loop usually succeeds on the second attempt, but a handler that fires at a high rate can starve it, and a loop with no bound is a hang with no diagnostic. The types and functions of the C memory model, _Atomic qualified objects and the compare-exchange operations from the standard atomics header, are the portable way to express the same thing; whether they compile to exclusive accesses or to a mask depends on the target, and a target that has no lock-free implementation for the type in question may not provide them at all. Other architectural caveats apply to exclusive accesses to device memory, as noted in read-modify-write hazards: an exclusive load followed by an exclusive store against a peripheral register is not a substitute for a set or clear alias.

Reentrancy, and drivers called from both contexts

A driver function that is reachable from the main flow and from an interrupt handler is the source of a class of bug that no critical section placement inside the main flow can fix. If the handler calls the function while the main flow is inside it, the handler sees a half-updated data structure regardless of what either caller does with masks: the mask that protects the main-flow call is set in the main flow, and the handler runs at a priority above it, if it runs at all.

The relation that determines whether a mask helps is between the masking threshold and the priority of the handler that touches the state. A mask set at a threshold above a handler’s priority blocks it; a mask at or below it does nothing. On a system with several handlers sharing one driver, the only reliable designs are to give the shared state a single writer and make the other contexts communicate with it through flags and queues, or to make the function reentrant by construction, so that it holds no state across a pre-emption point.

The classification cost is real: every function has to be labelled as callable from an interrupt context or not, and the label has to be honoured, which usually means the interrupt path uses a reduced variant that does not allocate, does not take a mask it cannot rely on, and does not call back into code that assumes it cannot be interrupted. The alternative, protecting every shared structure with the mask in all contexts, collapses the interrupt latency to the longest of those sections, which is the outcome the threshold register was supposed to avoid.

Ordering is a separate problem

Neither masking nor an exclusive store orders memory accesses as the hardware sees them. Both mechanisms stop another execution context on the same core from interleaving; neither pushes a store out of a write buffer, and neither prevents a device from seeing two stores in the order the interconnect preferred. Where the invariant being protected is about what the peripheral observes, an ordering instruction is needed in addition to the exclusion, either a data synchronisation barrier or a read of the register that was just written. Compiler-level ordering comes from the memory clobber or from an atomic with the appropriate ordering, and hardware-level ordering from the barrier. Conflating the two produces code that is correct in the abstract machine and wrong on the bus.

Choosing per invariant

The practical rule is to classify each shared piece of state by who touches it. State touched only by the main flow needs nothing. State touched by the main flow and one handler is protected either by a mask if the update is short, or by an exclusive retry if the mask would extend a latency budget, or by designing the update so it is a single aligned word write that cannot be torn. State touched by two handlers at different priorities is not fixable by masking from the main flow; it needs the priority relationship to be examined, because masking in the lower-priority handler does nothing against the higher-priority one. State read by more than one observer after the write needs ordering on top. The failure that results from getting the classification wrong is a value that is updated twice and read once, whose symptom is arithmetic that is off by a small amount at a rate that depends on load, and which appears only when two events happen to coincide.