Fixed-Width SIMD Lanes Versus Scalable Vector Registers

Every data-parallel instruction set must answer one question: how many elements does this instruction process? There are two coherent answers, and they produce two families of machines with different compilers, different binaries, and different failure modes. In the first, the answer is a constant written into the opcode or implied by the register name: a register is 128 bits, an instruction processes four 32-bit floats. In the second, the answer is held in a control register that software sets at run time, and the register file is a set of registers whose length the implementation chooses.

The distinction sounds cosmetic. It is not. It determines whether a binary compiled once runs efficiently on a machine with twice the parallelism, whether the compiler must unroll, how much register pressure a loop carries, and what a function call must save.

The register file as the contract

A fixed-width design fixes the number and width of vector registers in the ISA. A 32-register file of 128-bit registers, holding either four single-precision floats or two doubles, is the classic arrangement; a 32-register file of 256-bit or 512-bit registers is its extension. Because the width is in the encoding, the register file size in bytes is also in the encoding: 32 registers of 512 bits is 2 KiB of architectural state.

That 2 KiB figure matters for context switching. An operating system must save and restore the vector state of a thread on every context switch or lazily on first use. At 128 bits the state is 512 bytes, comparable to the general-purpose registers and cheap to save unconditionally. At 512 bits it is 2 KiB per thread, and a machine with many threads has a large aggregate state to move through memory. Implementations handle this with lazy save: a flag marks whether the vector unit has been used since the last switch, and the save happens only for threads that touched it. The mechanism works, but it means the cost of a context switch depends on the previous thread’s behaviour, which is a scheduling complication rather than a hardware one.

A scalable design inverts the relationship. The ISA defines vector registers of a length VLEN that is a property of the implementation, reported to software by a read-only register or a status field. Software writes a desired length, in elements, into a control register before a loop; the hardware clamps it to what the implementation supports and reports the value actually granted. The register file size is then a property of the implementation, and the operating system discovers it rather than assuming it. The cost is that the architectural state size varies across implementations, so save and restore code must be generated or dispatched per implementation, and any code that hard-codes a vector length is wrong on some machine.

How code generation differs

For the fixed-width machine, the compiler’s job is to map a loop onto a constant number of lanes. A loop over n floats on a four-lane machine becomes n/4 vector iterations plus a scalar remainder loop of up to three iterations. The remainder is unavoidable because n is not generally a multiple of four, and the compiler must emit both the vector loop and the scalar cleanup, or a masked version if the ISA provides masking. Unrolling by two or four vector iterations is common, purely to give the scheduler independent work and to amortise loop overhead.

For the scalable machine, the compiler emits one loop and asks the hardware how much it can do per iteration. A typical shape strips the loop by the granted length:

    # RISC-V vector style strip-mined loop, avl = n
    vsetvli t0, a0, e32, m1     # request VLEN/32 elements, get actual in t0
    vle32.v  v0, (a1)           # load v0 from a1, t0 elements
    vfadd.vv v1, v0, v2         # v1 = v0 + v2, t0 elements
    vse32.v  v1, (a2)           # store t0 elements to a2
    sub      a0, a0, t0         # remaining elements
    slli     t1, t0, 2          # bytes consumed = elements * 4
    add      a1, a1, t1
    add      a2, a2, t1
    bnez     a0, loop

The loop has no unrolled copies and no scalar remainder. The last iteration is handled by the hardware’s tail semantics: if fewer than t0 elements remain, the hardware either processes only the active ones or lets inactive lanes keep their previous values, depending on the tail policy configured. That single mechanism removes the cleanup loop entirely, which is a real reduction in code size and in instruction fetch.

The trade appears in the instruction count per element. On a 128-bit machine, one vector instruction handles four floats, and the loop overhead is amortised over four elements; on a 512-bit machine, one handles sixteen, and the same overhead amortises over sixteen. The scalable loop’s overhead per iteration is higher, because it must compute the granted length and update three pointers, and that overhead is paid on every iteration regardless of width. For short loops, the fixed-width machine’s simpler loop wins; for long loops, the difference is negligible.

Register pressure and the unroll factor

Fixed width forces the compiler to unroll to fill the pipeline, and unrolling consumes registers. A common pattern on a machine with sixteen vector registers keeps two to four accumulators per unrolled copy and needs the loop’s live values to fit; four-way unrolling with three accumulators per copy exhausts sixteen registers quickly. Spilling a vector register costs a full-width load and store, which is 128 bits of traffic per spill on the narrow machine and 512 on the wide one.

A scalable design has a fixed number of registers, typically 32, but their width grows with the implementation. The number of architectural registers does not change with VLEN, so a loop written for a small machine uses the same register names on a large one, and the compiler does not need a different unroll factor per implementation. This is the property that makes one binary portable across a family: the same loop uses the same registers and processes more elements per iteration on the larger machine.

The pressure is different in kind rather than absent. Because the hardware length is unknown at compile time, the compiler cannot allocate registers against a known number of live elements. It must be conservative, or it must generate code that adapts, which is what the control-register model permits: software can request a smaller length to reduce live state, trading iterations for registers. This knob has no analogue on a fixed-width machine, where the only lever is unrolling less.

What happens when the machine changes

Consider a binary compiled for a four-lane machine and run on a sixteen-lane machine. Every vector instruction in the binary processes four elements, because the width is in the opcode. The machine executes it as a single four-element operation and leaves twelve lanes idle. The extra hardware is wasted unless the binary is recompiled. Hand-written assembly and vendor libraries shipped as binaries have the same problem, which is why fixed-width families tend to introduce a new instruction set extension rather than widening the existing one: the wider form is a distinct encoding, and software selects it at run time by checking a feature flag.

A scalable binary on a wider machine requests a length and receives the larger one. The loop runs in fewer iterations. The same binary also runs on a narrower machine, in more iterations. No feature dispatch and no recompilation is needed for the common case, though code that wants the best possible schedule may still branch on the maximum length. The catch is that performance is not guaranteed to scale with width: the loop overhead per iteration, the memory bandwidth per cycle, and the number of outstanding loads all bound the gain, and a machine twice as wide with the same load bandwidth gains much less than 2× on streaming data.

There is a middle position. An ISA can define vector registers of a maximum width and specify that an implementation may support any width up to it, with the actual width reported at run time; software then uses the reported value. This keeps the register numbering and encoding stable while allowing a family of implementations. The design cost is that the maximum width sets the architectural state bound, and the minimum width the software must handle is often small, so the same binary may run at a quarter efficiency on a small implementation.

Choosing between them

The choice follows from what must be portable. If binaries are compiled for a known set of implementations and the vendor controls the toolchain, fixed width is simpler: the compiler knows the lane count, the scheduler can plan exact unroll factors, and the instruction encoding is compact because the width need not be encoded per instruction. If binaries must run across a family of implementations with different parallelism, the scalable model avoids recompilation and avoids the waste of idle lanes.

Fixed width also has an advantage in instruction encoding density for short vectors and in the availability of a large body of existing library code written against it. Scalable designs have an advantage in register-file utilisation and in the ability to express a loop once. Neither is universally better; they optimise different axes of the same trade, and both must solve the same downstream problems of masking and memory access, taken up in Mask Registers, Tail Handling, and the Cost of the Last Iteration and Strided, Indexed, and Gather Memory Access.