Chiplets and the Cost of Leaving the Die

A monolithic die holds every function on one piece of silicon, which makes all the wiring on-chip and all the bandwidth effectively free relative to the outside world. It also makes yield an exponential function of total area and forces the whole design through one process, one reticle field, and one tape-out. Partitioning a design into several dies attacks those constraints directly, and pays for it with an interface that has to carry across a package the traffic that used to stay inside a millimetre of silicon.

Whether the exchange is favourable depends on how much area the design can shed and how much traffic has to cross the new boundary. Both terms are quantifiable, and the interesting engineering is in choosing where to cut.

The yield argument, restated with numbers

The economics of area follow from the defect-limited yield model described in The Area Budget. If the probability that a die of area $A$ is free of defects falls roughly exponentially in $A$, then splitting one large die into $n$ small ones changes the expected number of working dies per wafer.

Consider a 600 mm² die against four 150 mm² dies, with the same defect density. Each of the small dies yields substantially more than the large one, so the number of good units per wafer rises even though the total silicon area is the same plus the scribe lanes between them. The gain is larger when defect density is high and smaller when the process is mature, which is why partitioning became attractive exactly when advanced-node defect densities stopped improving quickly.

There is a second effect that is often larger than the yield effect: different functions want different processes. Logic wants the fastest transistors; SRAM wants density and low leakage; analog and high-voltage interfaces want thicker oxides and larger geometries. A monolithic die forces one compromise across all of them. Separate dies let each function sit on the node that suits it, and a memory die at a mature node can be cheaper per bit than the same memory at the leading edge.

The third effect is reticle headroom. Once the design reaches the exposure field, the only way to grow is to add another die. Designs that expect to grow incrementally, such as a compute tile that is replicated to make a larger part, benefit from never having to face that wall.

What a die-to-die interface actually costs

The interface is where the accounting turns. On-die wires are cheap in energy per bit because they are short, narrow, and driven by local logic; a few hundred micrometres of metal in a modern process transports a bit for a small fraction of a picojoule at moderate voltage swing. Off-package links across a printed circuit board cost roughly an order of magnitude more, in the range of several to tens of picojoules per bit, because the capacitance and the required voltage swing are far larger. Die-to-die links in an advanced package sit between those two points: they are short enough to use narrow, low-swing signalling with no analog equalisation, and their energy per bit is commonly quoted at a small fraction of a picojoule to a few picojoules, depending on the density and the distance. Those figures are order-of-magnitude statements from the packaging literature, not properties of a particular product.

Three terms make up the cost.

Energy per bit, which sets the power of the link at a given bandwidth and therefore competes directly with the power budget of both dies.

Latency, which is the sum of the driver, the package trace, the receiver, and any retiming stages. A few hundred picoseconds of added latency is small compared with a DRAM access but large compared with an on-chip cache access, so the interface is affordable for traffic that was already slow and expensive for traffic that was fast.

Area, both for the link’s physical layer on each die and for the wiring channels in the package. The interface consumes silicon on both sides of the boundary, so the area saved by splitting must exceed the area spent.

# Whether a cut is favourable: area saved by splitting minus interface
# area, against the traffic that must now cross.
def cut_is_favourable(area_saved_mm2, phy_area_mm2, bits_per_second,
                      pj_per_bit_off_die, pj_per_bit_on_die, power_budget_w):
    net_area = area_saved_mm2 - 2 * phy_area_mm2      # PHY on both dies
    if net_area <= 0:
        return False, None
    link_watts = bits_per_second * (pj_per_bit_off_die - pj_per_bit_on_die) * 1e-12
    return link_watts < power_budget_w, link_watts

The change in energy per bit, not the absolute figure, is what the design has to absorb, because the traffic existed before the split. If a boundary carries signals that were previously routed a few millimetres on-die, the delta can be a large multiple, and the power of the link becomes a first-order term. If the boundary carries traffic that was already leaving the package, the delta is small or even negative, and the split is nearly free in energy.

Where to cut is the whole problem

The interface cost pattern suggests the rule: cut where bandwidth per unit area is low.

Memory is the natural first candidate. A DRAM interface already crosses a package boundary, so moving the memory controller to a separate die or placing the DRAM stacks in the same package adds little to a traffic stream that was already expensive to transport. That is why high-bandwidth memory packages place DRAM close to the logic die rather than remoting it. The logic-to-memory traffic is enormous in aggregate but it was already crossing a boundary, and the package shortens the distance.

Repeated compute tiles are the second candidate. If the same core complex is instantiated several times, splitting it from the shared uncore costs one interface per tile rather than one per wire, and the tile can be tested independently so that a defective tile does not condemn the whole part. The uncore’s traffic to each tile is comparatively modest and latency-tolerant compared with the traffic inside a tile.

What does not partition well is anything with a wide internal interface on the critical path. A monolithic out-of-order core has a register file with hundreds of bits of port width, a bypass network that spans the whole pipeline, and a load-store path whose latency directly sets performance. Placing a boundary anywhere inside that structure converts a short wire into a package link and adds the interface latency to the core’s critical path, which is the one place a few hundred picoseconds is decisive.

A second non-candidate is anything whose state must be kept consistent across the boundary at fine granularity. Coherence traffic between two halves of a last-level cache is latency sensitive and bursty, and every coherence transfer pays the interface. Splitting a cache works when the split is along address lines with little sharing, and works badly when the working set is interleaved.

Failure modes that only appear after partitioning

The dark side of a partitioned design shows up in the details.

The interface becomes a hard failure boundary. A link that fails to train prevents the package from booting at all, whereas an on-die wire that is defective can often be routed around or fused off. The testing budget grows accordingly, and parts that pass die-level test can still fail at assembly.

Binning becomes more complex. A package’s performance is the minimum over its dies: a fast compute die paired with a marginal memory die is sold as a marginal part, or not sold at all. The distribution of package performance is therefore worse than the distribution of its components, which is a real cost of assembling parts whose individual yields look good.

Thermal coupling crosses the boundary. Two dies in one package share a heatsink and leak heat into each other through the substrate, so the sustained power of the pair is not the sum of what each could sustain alone. Hot spots on one die raise the junction temperature of its neighbour, and the throttling controller must run a package-level policy rather than a per-die one. That is the same problem as Thermal Design Power at a larger scale, and it is one reason a partitioned design’s sustained operating point must be derived from the package rather than from the dies.

Latency asymmetry appears in software. Memory that is one hop away on the package and memory that is one hop away through an interconnect are not equally accessible, and code that performs well when the latency is uniform degrades when it is not. The scheduler and the memory allocator need to know which memory is near, which is the same problem as the one described in Heterogeneous Cores but with bytes instead of threads.

The trade, stated compactly

Partitioning converts an area-and-yield problem into a bandwidth-and-latency problem. It pays when the design has sections that can be cleanly separated, when the traffic across the resulting boundaries is low relative to the function on each side, and when the process requirements of the sections differ enough to justify separate tape-outs. It costs when the critical path crosses the boundary, when coherence or synchronisation traffic is high, and when the assembled package’s thermal and binning behaviour is worse than the sum of its parts.

The decision is not permanent in either direction. A design can start monolithic and split at the next generation once the traffic pattern is known from measurements on real silicon, which is the only source of the interface load figures the decision needs. Guessing the traffic mix is how partitioning projects go wrong, because the boundary that looks cheap on a block diagram is frequently the one carrying the most latency-sensitive traffic in the machine.