Acquire and Release Ordering: What a Lock Actually Guarantees About Memory

Two threads share a structure. One fills it in and then sets a flag. The other spins until the flag is set and then reads the structure. Written as plain C, the protocol is wrong on two counts. The compiler may move the store to the flag ahead of the stores to the fields, and the processor may make the flag visible to another core before the fields are visible, or reorder a load ahead of an older store. Neither failure requires an exotic machine: the first occurs with any optimizing compiler, and the second occurs on every architecture, x86 included, in at least one of its forms.

The compiler reorders before the hardware does

Consider the publishing side:

struct record { int a, b, c; };
struct record shared;
int ready;

void publish(void)
{
    shared.a = 1;
    shared.b = 2;
    shared.c = 3;
    ready = 1;          /* may be stored before the three above */
}

A compiler that can see publish in isolation may sink or combine the stores, and it may keep ready in a register until the end of the function while emitting the three field stores early. Nothing in the C abstract machine forbids this, because a single-threaded execution cannot observe the difference. Adding volatile to ready prevents the compiler from caching it in a register, but volatile says nothing about ordering relative to non-volatile accesses, and it emits no instruction that constrains the processor. The volatile keyword is a device-register tool, not a synchronization tool.

The consumer has the mirror problem:

void consume(void)
{
    while (!ready)
        ;
    use(shared.a, shared.b, shared.c);   /* loads may be hoisted above the loop */
}

A compiler may load shared.a before the spin loop terminates, since the load does not depend on ready in the abstract machine. That transformation is legal and produces a stale value even on a processor that never reorders anything.

Why a store is not immediately visible

The hardware half of the problem comes from the store buffer. A core does not write through to the coherence point on every store; it places the store in a queue, retires the instruction, and continues. The line is invalidated in other caches either immediately or when the buffer drains, depending on the protocol. The buffer is small, on the order of tens of entries on current designs, but it is deep enough for the publishing core to execute the store to ready while the three field stores are still queued.

The consuming core can then observe ready == 1 and read the fields from its own cache in the old state. The two cores agree that the flag is set and disagree about the data. This is not a coherence bug: the coherence protocol guarantees a single order per address, not a single order across addresses. A store buffer, a private cache, and an out-of-order pipeline each preserve per-location order while allowing cross-location reordering.

x86 processors implement a memory model usually called total store order. Stores are not reordered with other stores, loads are not reordered with other loads, and a load may be satisfied before an older store to a different address has drained. The practical consequence is that store-to-load is the only reordering x86 leaves open. That does not break the publish-and-consume pattern above, because the flag store cannot overtake the field stores and the consumer’s field loads cannot be hoisted above the flag load; it breaks the opposite shape, in which a thread stores to one location and then loads another, as the two-flag handshake below shows. A fence between the field stores and the flag store is therefore unnecessary for the pattern as written, and a fence on the consumer side is needed only when the consumer also stores before it loads.

Arm and RISC-V permit far more. Loads may be reordered with loads, stores with stores, and either with either, subject to data dependence and to explicit ordering instructions. The architecture provides acquire and release variants of the load and store instructions, LDAR and STLR on Arm, and the acquire and release bits on lr/sc on RISC-V, together with standalone fences such as DMB ISH on Arm and fence on RISC-V. A release store orders all prior accesses before itself; an acquire load orders all later accesses after itself.

Release and acquire as a pair of half-fences

The key insight behind the modern formulation is that publishing and consuming need opposite halves of a fence, and neither side needs the whole thing. The publisher needs everything before the flag store to happen before it: that is a release. The consumer needs everything after the flag load to happen after it: that is an acquire. Together the two halves form a synchronizes-with edge, and everything sequenced before the release in the publisher happens-before everything sequenced after the acquire in the consumer.

#include <stdatomic.h>

struct record { int a, b, c; };
struct record shared;
atomic_int ready;

void publish(void)
{
    shared.a = 1;
    shared.b = 2;
    shared.c = 3;
    /* Nothing above this line may become visible after the store. */
    atomic_store_explicit(&ready, 1, memory_order_release);
}

void consume(void)
{
    while (atomic_load_explicit(&ready, memory_order_acquire) == 0)
        ;
    /* Nothing below this line may become visible before the load. */
    use(shared.a, shared.b, shared.c);
}

The generated code is cheaper than a full fence on both sides. On x86 a release store is an ordinary store, because the hardware already orders stores with stores; an acquire load is an ordinary load, because loads are already ordered with later loads. On Arm the release store becomes STLR and the acquire load LDAR, each of which carries its ordering in the instruction rather than in a separate barrier. The saving relative to a pair of full barriers is the direction they constrain: a release store must not be visible before earlier accesses and an acquire load must not observe later accesses early, whereas DMB on either side would order both directions.

memory_order_seq_cst would also work, and is what a plain atomic store or load gets by default in C11 and C++11. It adds a single total order over all sequentially consistent operations, which costs an extra fence on some architectures and buys nothing for a two-thread publish-and-consume pattern. memory_order_relaxed is the other extreme: it guarantees atomicity and per-object coherence, and nothing else. A relaxed counter is fine; a relaxed flag guarding data is a bug.

Fences when the data does not flow through one variable

Sometimes the ordering edge cannot be attached to a single load or store. A kernel path that publishes several structures and then signals through an unrelated mechanism needs a standalone fence. The Linux kernel expresses this with smp_mb(), a full barrier that orders all memory operations on both sides, and smp_rmb()/smp_wmb() for the one-directional cases. The smp_ prefix matters: on a uniprocessor build these compile away, because a single core cannot observe its own reordering, while mb() remains for device-visible ordering. The compiler barrier barrier() is separate again, emitting nothing but forbidding the compiler from moving accesses across that point.

/* Kernel-style: publish, then raise a flag observed elsewhere. */
WRITE_ONCE(shared.a, 1);
WRITE_ONCE(shared.b, 2);
smp_wmb();                 /* stores above complete before the store below */
WRITE_ONCE(ready, 1);

WRITE_ONCE and READ_ONCE exist because the compiler may otherwise tear a wide access into several, or reload a variable it has already read. They are not barriers; they make one access indivisible and prevent the compiler from assuming it is the only writer.

The corresponding fence on the reader side must sit between the flag read and the data reads. Placing it after the data reads does nothing, and placing it before the flag read does nothing for this pattern. Half of all memory-ordering bugs are fences in the right place with the wrong extent; the other half are fences in the wrong place.

The pattern that needs a full barrier anyway

Release and acquire cover the case where the data flows through one flag. They do not cover store-to-load ordering, which arises whenever both threads write their own flag and then read the other’s. That pattern appears in two-thread handshakes and in Dekker-style protocols, and it is the one case where a half-fence on each side is not enough: each thread needs its own store to be visible before its own load executes, which is a full barrier in the middle of the thread.

x86 orders stores with stores, so the release half is a plain MOV and needs no instruction for the hardware’s part; the load half is a plain load too. What neither half constrains is a load overtaking an older store, so the store-load handshake needs a compiler barrier on each side, which forbids the compiler from exchanging the two accesses, and on x86 a hardware barrier as well (MFENCE, or any locked instruction, which on modern cores also orders a later load after an earlier store) because the store may still sit in the buffer while the load commits. Arm reaches the same effect with DMB ISH on each side, and LDAR/STLR alone do not supply it: each orders only in its own direction.

Where the model still surprises

A few consequences are worth stating because they are not obvious from the code.

Data dependence is not a substitute for an acquire on weakly ordered machines. A load of a pointer followed by a load through that pointer is ordered by the hardware’s dependence tracking on many cores, and the C11 model nonetheless requires an acquire operation or a consume dependency, because the compiler may speculate the second load. Relying on hardware dependence handling alone is a portability trap.

The synchronizes-with edge only exists if both sides use the annotations. A release store with no acquire load on the other side orders nothing for the consumer; the consumer may still see stale data. Conversely, an acquire load that reads a value written by a relaxed store does not synchronize with that store, whatever the store’s position in program order.

Finally, the ordering guarantee covers ordinary memory accesses, not I/O. A store to a device-mapped register needs an architecture-specific barrier, and a DMA buffer needs cache maintenance as well as a fence. The distinction between a compiler barrier and a CPU barrier, and between both and a device barrier, is the subject of endless confusion in driver code.

For the primitive that the fence protects, see atomic read-modify-write. For the case where a writer must not block readers at all, and the ordering problem becomes a retry loop rather than a fence, see seqlocks and optimistic readers. For the kernel’s answer to waiting that does not spin, see futex wait and wake.