What a Fence Instruction Actually Does to the Pipeline

A fence is the most misunderstood instruction in the memory system, largely because its name suggests an action on data. It does not flush caches, it does not write anything back, and it does not invalidate anything. It constrains the order in which operations become visible, and every cycle it costs comes from waiting for something to finish rather than from doing something.

Three distinct mechanisms appear under the name “fence”, and an implementation may use one, two, or all three depending on the ordering it must provide.

Mechanism one: draining the store buffer

The cheapest fence, and the one x86 uses for MFENCE, is a store-buffer drain. The core stops retiring stores into the buffer, waits until every entry has been accepted by the cache hierarchy, and then resumes. Nothing leaves the machine; the pending writes simply complete.

The latency is the depth of the buffer divided by the drain rate, bounded below by the oldest entry’s transaction. If the oldest store is waiting for a coherence transaction to complete, the fence waits for that transaction. In the common case where the buffer holds a few stores to lines the core already owns, the drain is short. In the worst case, where the buffer is full and the oldest entry is a store to a line held shared by several other cores, the fence waits for a full invalidation round trip.

Draining is sufficient for store-store and store-load ordering, because after the drain, no store is pending that could be observed out of order. It is not sufficient for load-load ordering on a machine that allows loads to complete out of order, since the fence does not wait for outstanding loads.

Mechanism two: waiting for acknowledgements

On machines with a weaker model, a fence must also wait for operations that have left the core but not yet become globally visible. A store that has been accepted by the local cache may still be in flight through the interconnect, and a fence that only drained the local buffer would not order it against a store issued later by the same core to a different destination.

The fence therefore waits for acknowledgements from the coherence fabric, and the number of acknowledgements depends on the topology. In a snooping design with a shared bus, the wait is until the bus has granted the request and any required invalidations have been acknowledged. In a directory design, the wait is until the home node has processed the request and replied. The cost scales with the number of sharers that must be invalidated, so a fence after a write to a widely shared line is more expensive than the same fence after a write to a private line.

This is the mechanism behind the observation that fence cost is a function of what the program was doing immediately before it, not a constant of the instruction. A fence on an idle core is cheap. A fence behind a burst of writes to shared lines is not.

Mechanism three: serialising memory operations

The strongest fences also cover loads and in-flight stores, not just the local buffer: no memory operation after the fence may become visible until every memory operation before it has completed. This is memory-operation serialisation, and it is what MFENCE on x86 provides beyond the store drain, and what DSB provides on Arm where DMB only orders.

Serialisation is expensive because it removes the overlap the pipeline was built for. A fence in the middle of a stream of independent work forces the core to wait for the oldest outstanding memory operation, which may be a load that missed in the cache. The cost is therefore not just the fence’s own latency but the stalls it creates around itself.

The distinction matters for the SFENCE, LFENCE, MFENCE family on x86. SFENCE orders stores against stores. LFENCE orders loads against loads and, on implementations that support the appropriate feature bit, also serialises instruction execution. MFENCE orders both loads and stores across the fence. Code that uses MFENCE everywhere is correct and pays its ordering cost at every site; code that uses the narrowest fence pays only for the ordering it needs.

Fences are not cache flushes

The flush misconception has a historical root: some early designs did provide instructions that wrote back and invalidated cache lines, and the word “barrier” was applied to both. The semantics are different in a way that matters.

A fence says: operations issued before this point are ordered before operations issued after it. It says nothing about where the data currently resides. After a fence, the line may still be in the local cache, dirty, and not yet written to memory. That is fine, because the ordering that matters is the order in which other cores observe the values, and a coherence protocol delivers the value from the cache that owns it. Writing it to DRAM would be slower and would change nothing about what other cores see.

A flush says: write this line back and drop it. That is a different operation with different purposes, such as making data visible to a device that bypasses coherence, or reclaiming cache capacity. The CLFLUSH instruction on x86 does this, and it is not a fence; it does not order anything except with respect to itself.

Confusing the two produces two opposite errors. Using a flush where a fence is needed gives no ordering guarantee, because the flush’s write-back may complete after a subsequent store. Using a fence where a flush is needed gives no visibility guarantee, because the device does not snoop the cache and the data may never reach memory.

What the cost actually is

The useful way to reason about fence cost is as a wait for the oldest outstanding operation, not as a fixed instruction latency. Three factors set it:

• The distance to the ordering point. A fence must wait until prior operations reach the point at which all observers agree on their order. In a bus-based machine that point is the bus; in a directory machine it is the home node; in a multi-socket system it may be a cross-socket link. Deeper hierarchies mean longer waits.
• The number of sharers to invalidate. A store to a line held in shared state by several caches requires an invalidation broadcast and acknowledgements from each. The fence behind it waits for all of them.
• The amount of work already in flight. A fence behind a long run of independent misses waits for the oldest of them. A fence behind a single store to a private line waits for almost nothing.

This is why fence cost is workload-dependent and why measurements that quote a single number for a fence are not transferable. It is also why the ordering is best established at the narrowest scope available: a fence that orders only within a thread, or only within a group of threads sharing a coherence domain, avoids waiting for the cross-domain acknowledgement entirely.

Narrower tools that avoid the fence

The reason fences are increasingly rare in compiled code is that narrower primitives carry ordering without a separate instruction. Three forms dominate.

A release store orders all prior operations against itself, at the cost of an instruction that carries a release attribute rather than a full barrier. An acquire load does the same in the other direction. The pair replaces two fences with two ordinary operations whose encodings carry the ordering, and on Arm these are single instructions rather than barriers. The pattern is described in Acquire and Release as One-Way Ordering Constraints.

A atomic read-modify-write with full ordering acts as a fence on both sides because it cannot execute until prior operations have drained and cannot retire until it is globally visible. A compare-and-swap loop therefore needs no separate fence around it, and this is why lock acquisition costs what it does: the atomic operation itself is the barrier.

A data dependency orders a load against a dependent load or store on some architectures without any fence, because the second operation cannot compute its address until the first has produced its value. This is real and load-bearing on Arm and PowerPC, where an address dependency provides ordering that the model would otherwise not give. It is also fragile: a compiler that breaks the dependency by recomputing the address, or a branch that replaces the dependency with a conditional, removes the ordering silently. Treating dependencies as ordering is a documented technique with documented hazards, covered in Read-Modify-Write Atomicity and the Ordering It Carries.

Placing a fence correctly

The placement rule follows from what a fence orders. It orders operations before it against operations after it, in the same thread. It does not order anything in another thread, and it does not order operations that the compiler has already moved across the fence’s position in the emitted code.

That gives three requirements for correct placement. The fence must sit between the two operations it is meant to order, in program order. The compiler must know that it may not move operations across the fence, which is a property of the language construct rather than of the instruction. And the fence must be strong enough for the ordering needed: a store-store fence does not order a load after a store, and using one there is a silent no-op.

The third requirement is where most fence bugs live. A store fence used where a load-store ordering is needed produces code that works on the machine it was tested on and fails on a machine with a different relaxation. The litmus tests in Litmus Tests and How Allowed Outcomes Are Decided exist to make that difference visible before it becomes a failure.