SIMD Encoding: Vector Register Fields, Predication and Length

A vector instruction does the work of many scalar instructions, so it must name more state than a scalar instruction does: several vector registers, an element width, a mask, and a length. None of that state fits in the thirty-two bits that a scalar format spends on three register numbers and an opcode. The encodings that solve the problem are therefore the place where an ISA’s field budget is under the most pressure, and the solutions chosen — extended prefixes, register-group numbering, implicit lengths — determine how much of the vector unit a compiler can actually reach.

Thirty-two vector registers in a five-bit field

A vector register file with thirty-two architectural registers needs five bits per register name. An instruction that reads two vectors and writes a third needs fifteen bits, which leaves seventeen for opcode, element width, mask, and everything else. That is not enough once the operation must also select among add, subtract, multiply, compare and their variants at four element widths.

x86’s AVX-512 encoding solves it by extending the instruction prefix rather than the instruction word. The EVEX prefix is four bytes and carries three additional register bits for each of the three operands, an opmask register number, a broadcast bit, a rounding control, and a two-bit vector length selector. The operation itself is still specified by the legacy opcode map, so the same 0F 58 that meant addps on a 128-bit register means a masked 512-bit add when preceded by the right EVEX prefix.

/* EVEX prefix fields, simplified: the register extensions are scattered
   across the prefix rather than contiguous. */
struct evex_prefix {
    uint8_t p0;   /* R X B R' 0 0 m m   */
    uint8_t p1;   /* W v v v v 1 p p     */
    uint8_t p2;   /* z L'L b V' a a a    */
};

The scattering is deliberate. The prefix’s bits were assigned to keep the decoder’s wiring compatible with the VEX prefix that preceded it, so the register extension bits sit where the earlier prefix’s bits already were and the two decoders share most of their logic. A new encoding that ignored the old one could have been more regular and would have required a second decoder.

The cost is that the instruction is long. A masked 512-bit operation with a memory operand and a displacement can occupy more than ten bytes, and the architectural limit on instruction length is fifteen bytes. Vector code is therefore the place where the length limit binds hardest, and where a compiler’s choice between a memory operand and a separate load has a measurable effect on the instruction stream’s size. The prefix budget itself is the same field-allocation problem that Opcode Space Allocation and How Extensions Claim Encodings describes at the byte level.

Register groups and implicit operands

An alternative to extending the prefix is to make the register number name a group. ARM’s Advanced SIMD encoding does this: a five-bit field names a vector register, and the instruction’s element width and operation determine how many consecutive registers the operation consumes. A 128-bit register is a pair of 64-bit registers in the same encoding space, and a widening operation reads a pair and writes a pair.

The group approach saves encoding bits by making the register count implicit in the operation. It costs the compiler the ability to place the elements of a group independently, and it costs the hardware the ability to rename a group’s members separately. A register allocator that works on the scalar register file cannot be reused unchanged, because a group is an allocation unit larger than a register.

The scheme also creates an aliasing problem that the architecture must define precisely. If a 128-bit register is architecturally the same state as two 64-bit registers, then writing the 128-bit register changes both halves, and reading a 64-bit half after a 128-bit write must see the corresponding half. The specification defines this as a single register file viewed at two widths, and implementations must ensure that a rename of the wide register is visible through the narrow view. Some implementations handle it by renaming the full width and extracting, which is cheap; others keep separate rename maps, which requires synchronisation.

Where the vector length lives

The length of a vector operation can be encoded in the instruction or held in a control register.

x86 encodes it partially. The EVEX prefix has a two-bit length selector that chooses 128, 256 or 512 bits, so the length is in the instruction but limited to three values. A loop over an array longer than the register still needs a scalar tail or a masked iteration, and the compiler must choose the length at compile time.

ARM’s SVE takes the other route. The vector length is a property of the implementation, held in a control register, and the instruction does not name it. The same instruction operates on whatever length the hardware provides, and the architectural register width is a maximum. Code written for SVE must be length-agnostic: it processes a multiple of the hardware’s vector length and handles the remainder with predication.

The trade is between compile-time knowledge and portability. A fixed-length instruction lets the compiler schedule and unroll for a known width, and it lets the register file be a fixed size in the encoding. A length-agnostic instruction lets one binary exploit a wider machine without recompilation, at the cost of a loop structure that cannot be fully unrolled at compile time and a set of predicates whose width is not known.

RISC-V’s vector extension uses a third model: the length is in a control register, but the register file is a fixed set of registers whose widths are implementation-defined, and the instruction encodes which registers participate through a nf field for register groups. A vector instruction names one base register and a field that says how many registers the group contains, which is a compromise between naming each register and naming a group implicitly.

Predication and the mask register

A masked vector operation suppresses the write to lanes whose mask bit is clear. The mask is architectural state, and the instruction must name it or use a fixed one.

AVX-512 encodes an eight-bit mask register number in the EVEX prefix and reserves mask register 0 to mean “no masking”. That convention is important because it lets the same opcode be used with and without masking without a separate encoding, at the cost of one mask register being unavailable for general use. The mask registers are a separate register file of eight registers, sixty-four bits wide, which is enough for the widest vector at the smallest element size: 512 bits divided by 8 bits per byte is sixty-four lanes.

ARM’s SVE makes the predicate a first-class operand with its own register file, and every instruction that can be masked names a predicate register in its encoding. The predicate registers are sized to the maximum vector length divided by the smallest element size, so a machine with 2048-bit vectors needs 256 predicate bits per register. The architecture defines the predicate as a register whose width scales with the implementation, which means the number of registers that fit in the encoding is fixed but their width is not.

The encoding cost of predication is a register field on every masked instruction. Architectures that cannot afford it restrict masking to a subset of operations, and the compiler must then generate unmasked code plus a fixup for the tail, which is exactly the code the mask was meant to remove.

Element width as an operand

The element width of a vector operation is usually encoded as a small field rather than derived from the opcode. A two-bit field selects byte, halfword, word or doubleword, and the same opcode with different widths performs the same logical operation on different data.

This is the encoding’s main source of density. One opcode plus a width field covers four instructions, and the operation selector covers the arithmetic. The alternative — a distinct opcode per width — would consume four times the opcode space for the same functionality, and vector opcode space is the scarcest resource in the architecture.

The cost is that the width field must be checked against the operation’s constraints. Not every operation is defined at every width: a saturating multiply may be defined only for halfwords, and a horizontal add may be defined only for words. The decoder must validate the combination, and the validation is a table lookup that grows with the number of operations. Architectures that define illegal combinations must also define what happens when one is executed, and the answer is usually an illegal-instruction fault, which is a precise trap that the compiler is expected never to trigger.

There is a second-order effect on the register file’s ports. A width field means the same register can be read at four widths, and the register file’s read path must extract the right sub-word. The extraction is a multiplexer, and its cost is proportional to the register width; a 512-bit register with four width modes needs a four-way select on every read port, which is a real area cost that a fixed-width scalar file does not pay.

Gather, scatter and the encoding of irregular access

A gather instruction reads several elements from non-contiguous addresses. Its encoding needs an index vector, a base register, a destination, a mask and a scale. That is five operands, and no format has five register fields.

The solutions are all compromises. x86 encodes a gather with a memory operand whose base is a vector register and whose index is another vector register, using the VSIB addressing form, and takes the mask from the instruction’s mask operand. The scale is a two-bit field in the SIB byte, so it is limited to 1, 2, 4 or 8, which matches the scalar element sizes. The destination is a vector register named in the ModR/M field. The encoding fits because the memory operand’s fields are reused for the vector index.

The performance of a gather is a separate matter from its encoding, but the encoding constrains the implementation. A gather with a mask that suppresses lanes must not fault on the suppressed lanes’ addresses, and the hardware must either check the mask before the address translation or translate all lanes and discard the results. The first option is faster and requires the mask to be available before the address generation, which is a scheduling constraint the encoding’s operand order does not express. A gather is also the only common instruction with a true two-register address form, the scaled-index mode described in Addressing Modes and How an Effective Address Is Computed, applied lane by lane.

What the encoding decides

The vector encoding determines three things that a compiler cannot work around: how many registers the register allocator can use, how wide an operation can be without a separate instruction, and whether masking is available without a branch.

A machine with thirty-two vector registers reachable only through a prefix has a register allocator that can use all thirty-two but pays for longer instructions. A machine with sixteen registers and a compact encoding has a shorter instruction stream and more spills. A machine with a length-agnostic encoding has portable code and a harder scheduling problem. None of these is a defect; each is the consequence of which resource the encoding chose to conserve, and the choice is visible in every loop the compiler emits.