Compiler Reordering and the Contract It Must Not Break

A fence in the instruction stream is a promise to the hardware. It is not a promise from the hardware’s point of view alone, because the code the hardware sees is not the code the programmer wrote. Between them sits an optimiser whose entire purpose is to move memory operations, merge them, delete them, and reorder them, subject to the constraint that a single-threaded program cannot tell. Concurrency is precisely the case where a single-threaded observer cannot tell but another thread can.

The language memory model exists to define which of those transformations remain legal when more than one thread observes the result, and to define what happens when a program gives the compiler no information to work with.

The as-if rule and its one-thread assumption

Every optimising compiler is licensed by a rule that says it may produce any program whose observable behaviour matches the source program’s, where observable behaviour is defined in terms of the program’s own execution: its return values, its output to volatile objects, and its calls into the environment. For a single-threaded program that licence is broad. A store to a variable that is never read again can be deleted. Two loads from the same address can be merged. A load can be hoisted above a store to a different address, because no single thread can observe the difference.

For a multithreaded program the licence is still that broad for data that is not atomic and not otherwise marked, because the language model says a program with a data race has undefined behaviour. The compiler’s obligation is then to the abstract single-threaded execution, and the definition of undefined behaviour is what makes the transformations safe from the compiler’s point of view. The consequence is that a program relying on the ordering of plain loads and stores across threads can be compiled into something with no ordering at all, and the failure is not a hardware timing issue but a source-level contract violation.

This is the single most important fact in this area, and it is routinely misread. The hardware model describes what a sequence of machine instructions may do. The language model describes what a source program may do. A program that is valid under the hardware model but violates the language model may be compiled into anything.

Atomic operations as compiler barriers

Atomic operations acquire their ordering from two separate mechanisms: the instruction the compiler emits, and the constraint the compiler places on its own transformations. The second is the part that is architecture-independent.

A release store is a barrier in the compiler’s model in one direction: it may not move earlier memory operations after it. Crucially, it may not move them after it even on a target where the hardware would not reorder them, because the compiler does not reason about the target’s hardware model when it applies the language model. It reasons about the abstract machine, and the abstract machine permits the reordering. A release store on x86 therefore compiles to a plain MOV followed by no barrier instruction, but the compiler still refuses to sink earlier stores below it. The ordering is enforced entirely at compile time.

An acquire load is the mirror: later operations may not move above it, and on x86 again no instruction is needed because the constraint is a compile-time one.

Relaxed operations carry no such constraint. A relaxed store may be moved, merged with another relaxed store to the same address, or eliminated if its value is never observed. This is not a deficiency; it is what makes relaxed atomics useful as counters and as statistics, where the ordering is irrelevant and the atomicity alone matters.

The pattern that most often surprises people is the loop:

// Busy-wait: the load must be re-issued on each iteration
while (atomic_load_explicit(&flag, memory_order_acquire) == 0)
    ;   // the atomic load may not be hoisted out of the loop

If flag were a plain int, the compiler would be entitled to hoist the load out of the loop and spin on a cached value forever, because nothing in the loop body modifies it and the single-threaded behaviour is identical. The atomic load makes the loop well-defined and forces a fresh load each iteration. The same pattern with a volatile qualifier instead of an atomic is a common but incorrect substitute: volatile prevents some of these transformations in practice for some compilers, but it provides no ordering and no atomicity, and neither the C nor the C++ standard gives it any synchronisation semantics.

Fences in the language versus fences in the instruction set

atomic_thread_fence is specified in terms of the abstract machine, not in terms of a target instruction. A release fence orders prior operations against subsequent operations in the abstract execution; whether that becomes an MFENCE, a DMB, or nothing at all depends on the target and on what the surrounding operations already guarantee.

Three consequences follow, and each one is a place where portable code breaks.

The fence’s position in the source matters, not only its presence. A release fence placed before the relaxed store it was meant to order produces a different abstract execution than one placed after, and the compiler is entitled to rely on that difference. The only guaranteed synchronisation is the one the standard defines for the specific arrangement of operations, and the arrangements that work are narrow. The discussion in Acquire and Release as One-Way Ordering Constraints covers the safe forms.

A fence can be stronger than any instruction the target needs, in which case the compiler emits nothing and the ordering comes entirely from the abstract model. It can also be weaker than what a naive reading suggests, if the target’s hardware model is weaker than the language model in some respect. The compiler’s job is to emit whatever is required to implement the abstract model on the target, and the mapping is not a one-to-one correspondence between source fences and machine instructions.

The reverse also occurs: a single source-level atomic operation can require several instructions on a weak target, and a source-level fence can require a sequence that includes a memory barrier plus a cache maintenance operation. The compiler is allowed to emit more than the minimum, and the programmer has no visibility into the mapping beyond what the implementation documents.

Where the optimiser interacts with hardware ordering

Optimisations that are legal in the abstract model can still produce surprising hardware behaviour when combined, and there are two named hazards worth understanding.

Load tearing and value speculation. A compiler may recognise that a load’s value determines a subsequent branch and speculate, or may split a wide load into narrower ones. For an atomic access the language forbids both, which is why atomic types have a guaranteed size and alignment and why the compiler emits a single access for them. For a plain access the compiler is free, and a plain 8-byte load on a 32-bit target becomes two 4-byte loads that another thread can observe separately.

Store merging. Two adjacent relaxed atomic stores to the same location can sometimes be merged, and two plain stores to the same location can always be eliminated except for the last. Merging changes how many times other threads observe the location change, which is legal for a relaxed atomic but not for a sequentially consistent one: the standard constrains sequentially consistent stores to a single total order that observers must agree on, and merging two of them would collapse two positions into one.

Dead store elimination. A plain store that is overwritten before being read can be removed if the compiler can prove it. In a concurrent program the proof does not hold, but the compiler does not need it to hold, because the program has a data race and therefore no defined behaviour. This is the case where the hardware model is completely irrelevant: no fence anywhere in the program prevents it.

The signal-handler-adjacent case

One boundary where the language model and the hardware model diverge in a way that affects ordinary code is between a thread and an asynchronous observer within the same program, such as a signal handler. The language’s atomic and ordering rules apply to threads; the guarantees for values shared with a signal handler are narrower, and a value that is atomic for threads may not be volatile enough for a handler to see. The practical rule, as stated in the standards, is that the only portable way to communicate with a handler is through an object of an atomic type that is lock-free, and even then the handler may only perform a restricted set of operations.

The hardware sees no difference. The compiler does, and the compiler’s rules are what the program must satisfy. A fence emitted because the programmer was thinking about the interconnect does nothing to persuade a compiler to keep a value in memory.

What the contract actually obliges

Putting the layers together, the obligation on a program is a single test: every access that is shared between threads is either atomic with sufficient ordering, or ordered against the other thread’s access by a synchronises-with edge that the language model recognises. Anything else is a data race, and a data race is undefined behaviour, which means the compiler owes nothing.

The programs that survive this test are the ones in which the fences and atomics are doing two jobs at once: constraining the optimiser to the abstract model, and constraining the hardware to the emitted instructions. The first job is mandatory everywhere. The second job’s cost varies by target, from nothing on x86 to a barrier on Arm, and the ranking of the costs is What a Fence Instruction Actually Does to the Pipeline.

Code that intends to be correct on both layers and is written without atomic types can usually be identified by a single search: plain loads of a flag inside a loop, plain stores of a pointer to initialised data, and plain increments of a shared counter. Each is a data race in the language model, regardless of what the hardware model would permit, and none of them is fixed by adding a fence instruction to the machine code rather than to the source.

The remaining complication is that the language model itself has evolved. The initial C++11 model forbade certain outcomes that some implementations produced and permitted others that later revisions restricted, and the treatment of consume-style ordering, which was meant to expose dependency ordering cheaply, proved difficult to specify and was deprecated in later revisions. Programs targeting a specific language revision inherit that revision’s model, and a claim about ordering that is true for one is not automatically true for another. That evolution is itself a design constraint: a memory model that is not implementable on real hardware, or not checkable by compilers, is not usable as a contract, and the revisions that followed the first attempt were largely about restoring implementability.