Forwarding Paths and the Load-Use Stall That Survives Them

A dependent instruction pair in a five-stage pipeline should not need to wait. The producer’s ALU result exists at the end of EX; the consumer needs it at the start of its own EX, which is one cycle later. Write-back happens two cycles after that. If the value travelled only through the register file, the consumer would have to stall until the producer’s WB, wasting two cycles per dependent pair in a sequence where such pairs are the common case. Forwarding closes the gap by wiring the result around the register file, and it works for every producer except one: the load, whose data arrives at the end of MEM rather than the end of EX. That single exception costs one cycle and is handled by a stall that the compiler and the scheduler both try to avoid.

The timing that makes a bypass necessary

Write the stage occupancy for a producer and an immediate consumer:

        cycle 1  cycle 2  cycle 3  cycle 4  cycle 5  cycle 6
add x1   IF       ID       EX       MEM      WB
sub x2   ...      IF       ID       EX       MEM      WB

The sub reads x1 in ID during cycle 3. At that moment the add is in EX and has not written anything. If the register file returns the old x1, sub computes the wrong answer. Note that the read happens in ID and the value is needed at the start of EX in cycle 4, so there is a full cycle of slack: the add completes EX at the end of cycle 3, and its result can be delivered to the sub’s EX input at the start of cycle 4 through a path that bypasses the register file entirely.

That is the EX/MEM to EX bypass. The same reasoning applied one step further gives the MEM/WB to EX bypass: a producer two instructions ahead writes back in the same cycle the consumer is in EX, and the value can be taken from the MEM/WB pipeline register rather than from the register file write port.

Both paths are multiplexers in front of the ALU’s operand inputs, selected by comparators that check whether the source register number of the instruction in ID matches the destination register number of the instruction in EX/MEM or MEM/WB, and whether that earlier instruction actually writes a register.

/* Operand selection for the EX stage. `idex` is the instruction now entering EX;
   `exmem` and `memwb` are the two older instructions still in flight. */
unsigned long operand_a(const struct idex_reg *idex,
                        const struct exmem_reg *exmem,
                        const struct memwb_reg *memwb,
                        const unsigned long *regfile)
{
    unsigned rs = idex->rs;
    if (exmem->reg_write && exmem->rd != 0 && exmem->rd == rs)
        return exmem->alu_result;        /* nearest producer wins */
    if (memwb->reg_write && memwb->rd != 0 && memwb->rd == rs)
        return memwb->mem_or_alu;
    return regfile[rs];                  /* no producer in flight */
}

The order of the tests matters. When both older instructions target the same register, the younger one holds the value that should be visible, so the EX/MEM path takes priority. Getting this backwards produces a result that is correct whenever dependencies are spaced out and wrong whenever they are tight, which is the kind of defect that survives casual testing.

Register zero is excluded from the comparison because it is hardwired to zero. Without the exclusion, an instruction with rd == 0 would appear to write the zero register and would forward a stale ALU result over the constant.

The load is the case forwarding cannot cover

A load’s data does not exist until the end of MEM. Trace a load and its immediate consumer:

        cycle 1  cycle 2  cycle 3  cycle 4  cycle 5
lw x1   IF       ID       EX       MEM      WB
add x2   ...      IF       ID       EX       ...

The add needs x1 at the start of its EX, which is cycle 4. The load’s data arrives at the end of its MEM, which is also cycle 4. There is no earlier point at which the value exists, so no bypass can supply it. The pipeline must hold the consumer in ID for one cycle, letting the load advance to WB, and then forward from MEM/WB to EX. One bubble, CPI contribution of one per such pair.

        cycle 1  cycle 2  cycle 3  cycle 4  cycle 5  cycle 6
lw x1   IF       ID       EX       MEM      WB
add x2   ...      IF       ID       ID(stall) EX      MEM

The detection condition is narrow and worth stating precisely: the instruction in ID is a consumer of the register that the instruction in EX is a load targeting. Everything else forwards. A load whose consumer is two instructions later is fine, because by then the load is in WB and the MEM/WB bypass applies. This is why a compiler that schedules an independent instruction between a load and its use eliminates the stall entirely, and why “load delay slot” scheduling exists on architectures that expose it.

The cost of the interlock is not just the bubble. Holding the consumer in ID while the pipeline advances means the IF stage must also be frozen, or the fetched instruction is lost. The usual arrangement freezes the PC and the IF/ID register for one cycle and injects a no-operation into the EX stage, which is what a bubble is: a control word with every write enable cleared. See pipeline interlocks and bubble injection for the general machinery.

What forwarding cannot fix and the register file must

Forwarding covers dependencies where the producer’s value exists somewhere in the pipeline. It does not cover a dependency separated by more than the pipeline’s depth, which is handled by the register file once write-back has occurred, and it does not cover a dependency where the consumer needs the value earlier than the producer produces it. The load case is the instance of the latter inside a single pipeline; across a deeper pipeline the same problem recurs at greater distance, because a producer several stages before EX has its result available even earlier relative to the consumer, but a producer in the last stage before WB may not.

A second gap is a producer whose result is computed in a unit that takes multiple cycles. A floating-point divide that occupies its unit for a dozen cycles cannot forward a partial result. The consumer stalls in ID until the divide signals completion, and the interlock holds it there for the full latency rather than one cycle. This is why the load-use penalty is described as a minimum, not a maximum.

There is also the question of which operand is affected. The load-use condition applies when the consumer reads the loaded register as either source. Store data is a special case: a store reads its data register in ID, but the value is not needed until MEM. Some designs add a MEM-stage bypass from MEM/WB directly into the store data path, which removes the need to stall a store whose data comes from the immediately preceding load. The instruction still occupies the pipeline, but the bubble is avoided.

Counting the cost in a real instruction stream

The relevant statistic is how often a load is followed within two instructions by a use of its destination. A compiler that knows the pipeline depth and the stall rule can reorder independent instructions into that gap; a compiler that does not, or code that cannot be reordered, pays.

If a fraction f of loads are followed immediately by a dependent use, the CPI contribution is f cycles per load. A stream with one load every five instructions and half of them followed immediately gives an added CPI of 0.1, which is a 10 percent throughput loss on an otherwise ideal machine. That number is not a measurement; it is the arithmetic of the stall rule, and it shows why scheduling is worth doing but cannot be the whole answer.

Two mitigations exist beyond scheduling. The first is to make the load’s data available earlier by moving address computation out of EX, so the load can issue its memory request a cycle sooner; this shortens the path but does not eliminate the gap. The second is to hide the gap with other work rather than with a bubble, which is the territory of superscalar issue and of dynamic scheduling: if an independent instruction is available, issue it into the slot the stalled consumer vacated, and the one-cycle penalty disappears without changing the pipeline depth.

The forwarding network itself has a cost that constrains how far it can be extended. Each additional bypass adds a multiplexer input to the critical path of the EX stage, and each comparator adds delay to the select logic. A pipeline with many in-flight stages needs a priority encoder over many candidates, and the multiplexer depth grows with the number of sources. This is one of the concrete reasons very deep pipelines do not simply scale forwarding to every stage: past a certain point the bypass network dominates the cycle time it was meant to protect.