Memory Channel Interleaving and the Granularity Trade

A processor’s memory controller does not see one large array. It sees several independent channels, each with its own command bus, its own data wires, and its own set of DRAM banks. Consecutive physical addresses are deliberately spread across those channels so that a stream of requests can keep all of them busy at once. The mapping that does this is fixed in the address decoder, and its parameters (the stripe size, the number of channels, the hash) determine which access patterns reach full bandwidth and which ones collide on a single channel.

Why one channel is never enough

The bandwidth of a channel is set by its data wires and its signalling rate. A 64-bit-wide DDR channel transferring on both clock edges at a few gigatransfers per second delivers on the order of ten gigabytes per second; the exact figure depends on the generation and the rate bin, but the shape is what matters. A core’s demand for a sequential stream can exceed that on its own, and a server with dozens of cores cannot be served by one channel regardless of how deep its queues are.

Adding channels adds bandwidth linearly in the number of channels, because each channel has independent wires and independent banks. That is the entire reason for the topology: four channels of the same width deliver four times the peak of one. The catch is that the extra bandwidth is only available if the request stream is spread across the channels, and spreading is not something the memory controller can do at request time — a request arrives with a physical address, and the address alone decides the channel. Interleaving is therefore a property of the address map, decided once at design time.

The stripe unit and what it costs

The unit of interleaving is normally the cache line, 64 bytes on the common architectures, or sometimes a 256-byte block. With line-granular interleaving, address bits above the line offset select the channel: the low bits choose a byte inside the line, the next bits choose a channel, and everything above that identifies a row and bank within the selected channel.

The consequence is that a single cache line’s fill is served entirely by one channel. A 64-byte line on a 64-bit channel is one burst, eight transfers of eight bytes, so there is no benefit to splitting a line across channels: the wires are busy for the whole burst either way, and splitting would require each channel to deliver a fragment of the same line, which is more command overhead for the same bytes. Line granularity is the natural unit because it matches the transaction the cache issues.

The alternative is finer interleaving, where the channel is selected by bits inside the line. That would let a single line’s fill be assembled from several channels in parallel, reducing the latency of one fill, but it requires the memory controller to reassemble fragments in order and to track partial lines. It also breaks the correspondence between a line and a burst, so each channel would issue a shorter burst with the same fixed command overhead. For these reasons, fine-grained interleaving appears only where latency of an individual access dominates, and standard server memory maps use line or block granularity.

Coarser interleaving (megabytes per channel) is used when the goal is not aggregate bandwidth but locality. If a channel maps to a pluggable DIMM and a process’s pages are placed on the DIMM nearest its core, coarse interleaving gives that process most of its traffic on one channel and leaves the others for other processes. This is what a non-uniform memory topology does: pages are placed in the memory attached to the core that will use them, and the channel that serves a page follows from which node’s memory the page was allocated in rather than from its offset within that allocation. Interleaving across all channels at line granularity is the uniform alternative: every core gets the same average bandwidth, and no process can be starved by another’s placement.

The hash, and why it is not a simple modulo

Selecting the channel by the low address bits above the line offset is the obvious scheme, and it is the one used by simple systems: channel equals (address >> 6) mod N. With a power-of-two number of channels the modulo is a mask of the low bits, which costs almost nothing in the address decoder’s critical path.

It has a serious weakness. Any access pattern whose stride is a multiple of N × 64 bytes maps to the same channel repeatedly. A program walking an array with a 4 KiB stride (one page at a time, common in page-table walks and in some matrix traversals) hits one channel on a four-channel machine and leaves the other three idle. Since the channels are independent, the machine delivers a quarter of its bandwidth for that pattern, and no amount of queuing depth fixes it because the bottleneck is the wires, not the banks.

Production address maps therefore hash higher address bits into the channel selection. A typical scheme XORs several groups of bits (often including bits from the row and bank fields) so that two addresses that differ only in a high bit still land on different channels.

/* Four-channel selection: XOR three bit groups above the 64-byte stripe. */
static inline unsigned channel_of(uint64_t pa)
{
        uint64_t line = pa >> 6;
        unsigned c = (unsigned)(line >> 0) & 3u;   /* bits 6..7   */
        c ^= (unsigned)(line >> 4) & 3u;           /* bits 10..11 */
        c ^= (unsigned)(line >> 12) & 3u;          /* bits 18..19 */
        return c;
}

The mapping must be deterministic and total, so that each physical address resolves to exactly one channel, rank, bank, row, and column, and it must be cheap, so it is usually a handful of XOR gates over a few bit positions rather than a cryptographic function. The exact bits are undocumented on most parts, which is why performance analysis of stride patterns on a specific machine is guesswork without a measured address map.

The hash also has to avoid aliasing with the DRAM internal structure. A bank is selected by a few address bits, and a channel has a limited number of banks; if the channel hash and the bank hash draw on the same bits, a stride that is meant to spread accesses across banks also shifts them across channels, so the two levels of parallelism stop being independent. Designers choose hash inputs to decorrelate the two, and this is one reason the address map is not simply “low bits to channel, next bits to bank”.

Bursts, bank groups, and the scheduling window

Interleaving at line granularity makes the request stream channel-parallel, but each channel still has to schedule its own accesses against DRAM timing. A DRAM row must be opened before a column can be read, and a row occupies the bank’s sense amplifiers until it is closed or another row is opened in the same bank. A channel receives only its share of the lines in a sequential stream, and those lines are not adjacent in physical address. They usually fall within a row that the channel already has open, so the stream produces an activation followed by several column accesses in that channel before the row has to be closed. The activate cost is amortised over fewer columns than it would be without interleaving, but it is amortised.

The stripe size decides how much work each visit to a channel carries. A stripe smaller than a cache line splits one line across channels, so each channel issues a short burst for the same fixed command overhead and the controller has to reassemble fragments; that is why the stripe is not taken below a line. A larger stripe, 256 bytes or a kilobyte, keeps several consecutive lines on one channel and raises the number of column accesses per activation, which helps row locality. The cost is that a short sequential run reaches fewer channels, so the first accesses of a stream see less parallelism. The granularity is therefore a choice about which access pattern the machine is built for.

Modern DDR parts add bank groups: the banks are partitioned into groups that can overlap certain timings, so a controller can issue a column command to one group while another group is still recovering. Exploiting that requires the controller to have several requests in flight to the same channel but different groups, which in turn requires the interleaving to deliver enough requests to each channel to fill the scheduling window. A stream that maps entirely to one channel can still saturate it, but a stream spread too thinly — for example a random access pattern where each channel sees one request at a time — leaves the bank groups idle and delivers latency rather than bandwidth.

What the mapping does to software

Three practical consequences follow. First, single-threaded sequential bandwidth is capped by the number of channels only if the core can issue enough outstanding misses; a core with a small number of fill buffers may not have enough requests in flight to occupy four channels, so adding channels helps multi-core aggregate bandwidth more than single-thread speed. Second, any stride that is a large power of two is a performance hazard, and the mitigation is to pad data structures so that the stride is not a multiple of the interleave period — the same reasoning that motivates padding to avoid cache-set conflicts, applied one level down.

Third, the mapping is not visible to software. There is no architectural register that reports the channel of an address, and the hash is not documented. A program can infer it only by measuring, and a measurement on one machine does not transfer to another with a different channel count or a different hash. The portable conclusion is structural: power-of-two strides and page-aligned access patterns are the ones to suspect, and the fix is to break the alignment rather than to compute the map.

Degenerate address maps

The mapping’s worst case is a channel that is populated but not enabled, or a channel whose DIMM is smaller than its siblings. A memory controller that interleaves across four slots when only three are populated will produce holes: addresses that map to the empty channel return errors or are remapped, depending on the platform, and the remapping changes the effective bandwidth of the rest. Platforms therefore describe the populated topology in firmware tables, and the interleave mode is chosen to match — a single rank per channel, matched capacities, and matched speeds are the conditions under which full interleaving is available.

A second failure is the interaction between interleaving and memory encryption or compression, both of which can change the effective address before the channel hash is applied. If encryption operates on a block size that is not aligned to the stripe, a single encrypted block may span two channels, and the controller must either serialise the two halves or handle partial bursts. If compression changes block sizes, the physical address of a compressed block no longer corresponds to its logical offset, and interleaving must be applied to the physical placement rather than the logical address.

A third is thermal and power: interleaving spreads traffic evenly, which is good for bandwidth and bad for the ability to power down idle channels. A uniform stream keeps every channel active, so no channel can enter a low-power state; a topology-aware placement that concentrates traffic on one channel can leave the others idle and save power, at the cost of bandwidth for that process. The two goals are in direct conflict, and which one wins depends on whether the machine is optimised for throughput or for energy.

How the requests reach these channels from a device rather than a core is a different path entirely, described in DMA addressing and the IOMMU; how a device’s writes are ordered once they arrive is in posted writes and ordering rules.