The Five-Stage Pipeline and the Throughput It Buys

A processor built as one long combinational path has to finish the slowest instruction before it can start the next one. For a load/store instruction set the slowest instruction is a load: fetch the instruction, read the register file, compute the address, wait for memory, write the result back. The clock period must cover that whole chain, so every add pays the load’s memory latency. Splitting the path into stages with registers between them does not make any instruction finish sooner. It lets five instructions be in the machine at once, each in a different stage, so one instruction retires per cycle in steady state instead of one per five cycles.

That is the entire argument for pipelining, and everything else in this cluster is a consequence of the fact that the five instructions in flight are usually not independent.

Why the single-cycle clock is set by the worst case

Take a conventional RISC datapath with separate instruction and data memories: an adder for the PC, an instruction memory, a register file with two read ports and one write port, an ALU, a data memory, and the multiplexers that steer results. In a single-cycle design all of that sits in one clock period, and the period is the sum of the delays along the longest path, which is the load path: instruction memory access, register read, ALU, data memory access, register write setup. If instruction memory takes 200 ps, the register file 100 ps, the ALU 150 ps, and data memory 200 ps, the path is roughly 650 ps plus multiplexer and wiring delay, and that is the clock period for every instruction including the ones that touch no memory at all.

A multi-cycle design attacks this by allowing an instruction to occupy several cycles, which lets the clock shrink to the longest single stage. Throughput does not improve, because an instruction now takes five short cycles instead of one long one. The resource that was idle in the single-cycle design was the hardware itself: while a load waited on memory, the ALU sat unused. Pipelining recovers that idle time by placing a register after each stage and letting a different instruction occupy each stage in each cycle.

The five stages and what crosses between them

The canonical decomposition, inherited from the MIPS design and used in most textbook treatments, is:

• IF fetches the instruction at the PC and computes PC+4.
• ID decodes the instruction, reads both source registers, and sign-extends the immediate.
• EX performs the ALU operation or computes an address.
• MEM accesses data memory for loads and stores.
• WB writes the result into the register file.

The pipeline registers are not incidental. Each one holds everything the following stage needs and nothing it does not, and the width of those registers is a real cost: the IF/ID register holds a 32-bit instruction plus a 32-bit PC, the ID/EX register holds two 32-bit register values, a 32-bit immediate, and the control bundle, and so on. A five-stage 32-bit pipeline spends on the order of a few hundred bits of state per stage boundary, and that state must be reset, scanned for test, and clocked every cycle.

The control signals deserve separate mention because they travel with the instruction rather than being regenerated in each stage. Decoding produces the full control bundle in ID, and each stage carries forward only the subset it still needs. By WB only two bits remain: register write enable and the select between the ALU result and the load value.

/* The steady-state occupancy of a five-stage pipeline for a straight-line
   sequence with no hazards. Each row is one instruction, each column a cycle. */
enum { IF, ID, EX, MEM, WB, NSTAGE };
static const char *name[] = { "IF", "ID", "EX", "MEM", "WB" };

void schedule(int n_insn)
{
    for (int cycle = 0; cycle < n_insn + NSTAGE - 1; cycle++) {
        printf("cycle %2d:", cycle + 1);
        for (int i = 0; i < n_insn; i++) {
            int stage = cycle - i;              /* instruction i starts at cycle i */
            if (stage >= 0 && stage < NSTAGE)
                printf(" %-3s", name[stage]);
            else
                printf(" %-3s", "-");
        }
        putchar('\n');
    }
}

For ten instructions this prints five cycles of ramp-up, five cycles of full occupancy, and five cycles of drain. The fill and drain are why pipelining is a bet on long runs of instructions: a five-instruction program gains nothing at all from a five-stage pipeline, and a program with a branch every six instructions spends much of its time refilling.

What the speedup is, once overhead is counted

Ignoring hazards, the ideal speedup over the single-cycle machine is the number of stages. Real speedup is smaller for two reasons that are visible in the arithmetic.

First, the register between stages has its own delay: setup time, propagation delay, and clock-to-Q. If a stage’s combinational logic takes t and the register costs t_r, the pipelined clock period is t + t_r rather than t. With five equal stages of 200 ps and a 20 ps register, the period is 220 ps and the throughput ratio against a 650 ps single-cycle machine is about 3, not 5.

Second, stages are not equal. The clock period is set by the slowest stage, so a datapath whose memory stage takes twice the ALU stage wastes the difference in every other stage. If the five stages take 150, 100, 300, 250, and 100 ps, the period is 300 ps plus register overhead and the ideal fivefold gain collapses toward a factor of two. Balancing stage delays is a design activity in its own right, and it is the subject of retiming and stage balancing. The same arithmetic, run in the other direction, explains why adding stages eventually stops paying: see deep pipelines and diminishing returns.

A useful way to state the result is in terms of instructions per cycle. An unpipelined machine completes one instruction every k cycles, so its CPI is k and its IPC is 1/k. The five-stage pipeline has an ideal CPI of 1 and an ideal IPC of 1. Every hazard that follows raises the CPI above 1, and the rest of the cluster is about which hazards do that and how much each costs.

Two structural details that make the fifth stage possible

The write-back stage appears to create an impossible conflict: in the same cycle, one instruction writes a register in WB while another reads two registers in ID. The register file resolves this without duplication by using the clock’s two edges: writes are triggered on the falling edge (or in the first half of the cycle) and reads are sampled on the rising edge, so a value written in the first half is visible to a read in the second half. The consequence for software is that a value produced by an instruction and consumed by the immediately following instruction can be read from the register file without a stall, provided the producer’s WB occurs before the consumer’s ID. When it does not, forwarding supplies the value instead, which is where forwarding paths enter.

The second detail is memory. A single memory used for both instruction fetch and data access would be busy in IF and again in MEM, and two instructions in different stages would collide. Splitting the memories, or at minimum giving instruction fetch its own port and its own small cache, removes the conflict. That is a structural hazard resolved by spending area, and it is one instance of the general pattern covered in structural hazards and resource conflicts.

Where the fixed five stages stop fitting

The decomposition above assumes fixed-length instructions, a register file with enough ports, a single-cycle ALU, and memory access confined to one stage. Each assumption fails somewhere. A variable-length instruction set such as x86 cannot decode in one stage, so real implementations insert pre-decode and length-decoding stages ahead of the classic five, which is one reason those pipelines are deeper than five. A floating-point multiply that takes several cycles cannot occupy EX for one cycle, so it either stalls the pipeline or runs as a separate unit with its own interlock. A cache miss in MEM cannot complete in one cycle either, and the pipeline must either stall or, in a non-blocking design, continue with other instructions while the miss is outstanding.

The five-stage pipeline is therefore best understood as the minimum viable overlap: the smallest structure that turns a serial instruction stream into a throughput-oriented one. Every later refinement in this cluster either removes a reason the ideal CPI is not reached, or buys more throughput by widening and deepening the structure at a cost that grows faster than the benefit.