The Universal Scalability Law: Why Throughput Peaks and Then Falls

Amdahl’s law describes a curve that rises and flattens. Systems in the field tend to rise, flatten, and then decline: a server with 64 cores serves fewer requests per second than the same server with 24, and the extra cores make the answer worse rather than merely useless. A model that cannot produce a declining region is describing the wrong shape, and the practical consequence is that capacity work on such a system targets a peak that the model says does not exist.

The universal scalability law adds one term to Amdahl’s model and acquires the peak. The added term accounts for the cost of keeping shared state consistent, which grows with the number of participants rather than staying fixed, and it is that growth which eventually bends the curve downward.

Two terms and where they come from

The law expresses relative throughput or capacity as a function of the number of processors or concurrent users $N$:

$$C(N) = \frac{N}{1 + \alpha(N - 1) + \beta N(N-1)}$$

The $\alpha$ term is the serial fraction in Amdahl’s sense: work that must be done by one entity at a time, expressed as a fraction of the time, giving a curve that approaches $1/\alpha$. The $\beta$ term is the new one. It is quadratic in $N$, which is what makes the peak possible, and it represents pairwise interaction: every participant must exchange something with every other participant, and the number of such pairs is $N(N-1)/2$.

The physical justification for the pairwise term is the coherence event. When $N$ threads share a piece of writable state, each write potentially invalidates the copies held by all other threads, and the cost of the invalidation is charged to whoever holds those copies. The number of affected participants grows with $N$, so the total cost of keeping the state consistent grows with $N$ even if each individual write is constant. A single mutex already exhibits this: its contention grows superlinearly because waiting threads slow the holder, and the holder’s slowdown slows everyone.

Parameter	Source of the cost	Growth	Consequence
$\alpha$	Serialized portion, one-at-a-time work	linear in participants	curve flattens at $1/\alpha$
$\beta$	Pairwise consistency, invalidations, retries	quadratic in participants	curve peaks, then falls
$\beta = 0$	No shared writable state	Amdahl’s law exactly	monotone approach to $1/\alpha$

The three rows are the reason the model is useful as a diagnostic rather than only as a prediction. A fitted $\beta$ of zero says the system has no pairwise consistency cost and Amdahl’s model is adequate. A fitted $\beta$ that is not small says the system will have a peak, and the peak’s location is a property of $\beta$ rather than of the hardware.

Locating the peak

The maximum of $C(N)$ occurs where the derivative is zero. Setting it up in the continuous form and solving gives

$$N_{\max} = \sqrt{\frac{1 - \alpha}{\beta}}$$

and the capacity at that point is

$$C(N_{\max}) = \frac{N_{\max}}{1 + \alpha(N_{\max} - 1) + \beta N_{\max}(N_{\max}-1)}$$

The location depends only on the ratio of the two parameters. A system with $\alpha = 0.02$ and $\beta = 0.0001$ peaks near $N = 99$; the same $\alpha$ with $\beta = 0.001$ peaks near $N = 31$. Both are perfectly plausible values for a service with a shared database and a shared cache, and the difference between them is the difference between a machine that uses its cores and one that is over-provisioned.

The peak is also shallow near its top and steep on the falling side. Capacity at $N_{\max}$ and at $2N_{\max}$ differs by a large amount, because the quadratic term dominates once past the peak. That asymmetry matters for capacity planning: running just past the peak is nearly harmless, and running well past it is much worse than running at half the size.

def usl(n, alpha, beta):
    return n / (1 + alpha * (n - 1) + beta * n * (n - 1))

def peak(alpha, beta):
    import math
    n = math.sqrt((1 - alpha) / beta)
    return n, usl(n, alpha, beta)

for a, b in ((0.02, 0.0001), (0.02, 0.001), (0.1, 0.0005)):
    n, c = peak(a, b)
    print(f"alpha={a:5.3f} beta={b:6.4f} -> peak near N={n:5.1f}, C={c:5.2f}")

The numbers produced by this script are model outputs, not measurements. Their value is in showing how sensitive the peak’s location is to a parameter that is hard to observe directly.

Fitting from data that already exists

The model has two free parameters, so two measurements in the scaling region suffice. Any pair of $(N, C(N))$ points with $N > 1$ can be solved for $\alpha$ and $\beta$, and more points overdetermine the system, which is what makes least-squares fitting both possible and a check on the model’s adequacy.

The measurements must be of throughput or capacity at a fixed load level, taken to steady state, on the same workload, with the thread or process count as the only variable. This is where most fits go wrong: a load generator that is a closed loop with a fixed client count does not hold the load fixed as $N$ changes, so the measured points are not samples of one curve. An open-loop generator at a fixed arrival rate, or a fixed-backlog throughput measurement, produces points on the curve.

With a good set of points, the fitting procedure is straightforward: express the equation as a linear problem in $1/C$ and inspect whether the resulting residuals are structureless. Rearranging gives

$$\frac{N}{C(N)} - 1 = \alpha(N-1) + \beta N(N-1)$$

which is linear in $\alpha$ and $\beta$ with known regressors. A linear least-squares fit gives both parameters, and residuals that trend with $N$ indicate that neither the two-term model nor the measurement setup is right.

The diagnostic value is at least as large as the predictive value. A system whose points fit the model well and whose fitted peak is below the configured thread count is running in retrograde. A system whose points fit Amdahl’s law with $\beta$ indistinguishable from zero has no pairwise consistency cost and the concurrency can be raised without a peak. The distinction between those two outcomes changes what to do next: in the first case, remove sharing; in the second, add parallelism.

What retrograde scaling looks like from inside

The falling region of the curve has an internal signature that is easy to miss, because each component appears to be working. Utilization is high, the queue depth is nonzero, and no component reports an error. What has changed is that an increasing share of each operation’s time is spent waiting for other operations rather than doing work.

Three symptoms distinguish it. First, the retry and conflict counts rise with thread count while the completed work stays flat or falls; these include failed compare-and-swaps, lock contention counts, and transaction aborts, all of which are direct measures of pairwise interaction. Second, per-thread instruction counts rise without per-thread work rising, which is the same observation in a different counter. Third, and most telling, reducing the thread count increases throughput, which no other diagnosis produces.

The interventions follow the model’s structure. Lowering $\alpha$ means shortening or removing serialized work: coarser critical sections merged into fewer, per-thread state replacing shared state where the semantics allow it. Lowering $\beta$ means removing pairwise consistency: partitioning the shared state so that no two participants contend on the same location, replacing shared writable data with replicated read-only data, or batching so that the number of interactions falls.

Batching deserves emphasis because it reduces $\beta$ directly rather than by removing functionality. If a thousand operations become ten exchanges, the pairwise cost per operation falls by two orders of magnitude and the peak moves outward by an order of magnitude. The technique is described in Batching and the Amortization of Per-Item Cost, and its purpose in a scalability model is precisely this parameter. Similarly, partitioning into per-thread state is the direct attack on $\beta$, as in Sharded Counters, Padding, and False Sharing.

Limits of the model

The law is a two-parameter rational function, and it is remarkable how often real systems follow it. It is still a description of a homogeneous workload on a homogeneous machine, and it has nothing to say about the shapes of individual latency distributions, about the effect of different work types sharing a pool, or about scheduling policy.

Two specific caveats. The model assumes that the contention and coherency costs are the same for all pairs of participants; on a NUMA machine they are not, and the cross-socket pairs cost more, which flattens the effective curve in a way a single $\beta$ absorbs as a larger value. And the fit describes the steady state at a particular load level; the parameters change when the workload changes, so a single fitted peak is not a permanent property of the system.

Used with those caveats, the model answers the question that capacity work actually asks: not how much faster a system could be in principle, but at what point additional concurrency stops helping and starts harming. The answer is a number computed from two measurements, and the fact that it is finite is the part that Amdahl’s law cannot express. The concurrency that the $\beta$ term charges for is the kind that adds in-flight work without adding parallel execution, and the bill for it arrives at the peak.