Reserved Versus On-Demand Capacity: The Break-Even Utilization

The choice between paying for capacity by the hour and committing to capacity in advance is a bet on utilization. A commitment buys a discount on every hour of the term; an on-demand arrangement buys flexibility at a higher hourly rate. If the committed capacity is used for most of the term, the discount dominates. If it sits idle for a large fraction of the term, the commitment costs more than the flexibility it replaced. The break-even is a single number, and it is not the utilization at which the two prices are equal in a given hour but the utilization averaged over the term.

The arithmetic of a commitment

Let $p_o$ be the on-demand hourly price of one replica and $p_r$ the reserved hourly price, with $d = p_r / p_o < 1$ being the discount factor. A commitment of $c$ replicas for $T$ hours costs $c p_r T$ whether or not the replicas are used. The same capacity, bought on demand for $u$ fraction of the hours, costs $c p_o u T$. The commitment wins when

$$c p_r T < c p_o u T \quad \Longleftrightarrow \quad u > \frac{p_r}{p_o} = d$$

So the break-even utilization is exactly the discount factor. At a discount of 40 percent, a commitment is cheaper than on-demand if the capacity is genuinely needed for more than 60 percent of the term. That result is clean and easy to state, and almost every practical difficulty lies in estimating $u$ correctly.

Two adjustments make it less clean. Committed capacity is not free to leave: a term is a term, and a workload that shrinks mid-term has no recourse. And committed capacity cannot exceed what a single commitment granularity allows, so the comparison is usually between a coarse block and a fine-grained stream, not between two continuous quantities.

A third adjustment comes from the utilization estimate itself. The $u$ in the inequality is the utilization of the committed block over the term, which is a different quantity from the utilization of the pool, because the pool includes elastic capacity that the block does not cover. A block that covers the overnight floor is fully utilized for the whole term even if the pool’s overall utilization is half that, and a block sized to the peak is underutilized for most of it. The two readings differ by the ratio of floor to peak, which for a diurnal service is routinely a factor of two or more. Sizing from the pool-level figure rather than the block-level one therefore tends to produce commitments that look justified and are not, a mistake the autoscaling lead time analysis makes visible when it computes how much capacity must be resident before a ramp.

Estimating utilization from a load curve

The load curve of a token service is not flat. Demand rises through a working day, peaks in the late morning or early afternoon, falls overnight, and repeats with variation across days of the week. The quantity that matters is not the mean utilization over the term but the fraction of hours in which the committed capacity would have been used. Those differ when the curve is spiky: a service whose mean utilization is 50 percent but which sits at 90 percent for six hours a day has a per-hour utilization pattern that supports a commitment sized to the peak and not to the mean.

The right procedure is to sort the load curve and read off the exceedance. If the demand in hour $h$ is $D_h$ and a commitment of $c$ replicas covers $\min(D_h, c)$, then the utilization of the commitment is $(\sum_h \min(D_h, c)) / (c T)$. This is a function of $c$, not a constant: committing to more capacity lowers the utilization of each committed replica even as it covers more hours. The optimization is therefore over $c$, and the objective is total cost including both committed and on-demand hours:

$$\text{cost}(c) = c p_r T + p_o \sum_h \max(D_h - c,\ 0)$$

This function is convex in $c$ because the first term is linear and the second is a sum of hinge functions, each convex. It has a minimum at the $c$ where the marginal saving from covering one more hour of demand equals the marginal cost of one more committed replica. Since the hourly demand is discrete, the minimum is at the quantile of the demand distribution where the fraction of hours above $c$ equals the ratio of prices. In the simplest form, with all hours weighted equally, the optimal commitment covers the demand at the $(1 - p_r/p_o)$ quantile of the hourly distribution.

The interpretation is worth stating plainly. If reserved capacity costs 60 percent of on-demand, commit to a level that is exceeded in 40 percent of hours. That level will look conservative against a peak-based plan, and it is correct, because the hours above it can be served on demand at the higher rate and the hours below it are cheaper for having been committed.

The three-way choice with interruptible capacity

The two-way comparison omits a third option that changes the answer: interruptible capacity, priced below both, available only when the provider has spare hardware, and subject to reclamation with short notice. Its economics are attractive for work that can be restarted, and unusable for work that cannot. The relevant question is not the price but the cost of an interruption, which for a token service is the wasted prefill and the client-visible failure.

Interruptible capacity suits batch or offline work where a reclaimed instance loses at most the tokens generated since the last checkpoint. It is a poor fit for interactive streaming, where a reclamation mid-response terminates a user-visible stream. A middle position is possible: interruptible replicas for the burst above the committed baseline, with admission control that routes interactive traffic to the stable pool and defers batch traffic to the interruptible one. This partitions the workload by latency tolerance rather than by price, and it makes the interruptible tier’s utilization independent of the interactive tier’s.

The break-even between tiers then becomes a comparison of three prices weighted by three utilization fractions, and the committed baseline should be sized to the floor of the load curve rather than to its quantile. A floor-based commitment is nearly always fully utilized, so its effective price is the reserved rate; the burst above the floor is served by whichever of the two elastic options is cheaper at the time.

Committed capacity is also a latency decision

Price is not the only term. On-demand and interruptible capacity have to be acquired, configured, and warmed before they serve traffic, and the acquisition time is on the order of minutes for a new replica that must load weights and initialize. A pool that relies on on-demand capacity to cover a peak must either acquire before the peak arrives, which requires forecasting, or absorb the cold-start delay in the latency target.

Committed capacity has no such delay because it is already resident. The value of that is highest for workloads whose peaks are sharp and whose latency targets are tight, and it is close to zero for workloads that can queue for minutes without consequence. The autoscaling analysis treats this as a separate term; here it is enough to note that a commitment can be justified at a utilization below the price break-even if the alternative is a pool that cannot be ready in time.

There is a symmetric cost on the committed side: a commitment sized to the peak of a workload that then grows past it produces a hybrid in which the committed block is fully utilized and the elastic overflow is expensive. Commitments that are sized to a conservative quantile and re-evaluated as the load curve is observed are less exposed to this than commitments sized to a forecast peak.

What the calculation cannot see

Three things fall outside the arithmetic. Price schedules change, and a commitment made at one discount may be repriced at renewal; the comparison is over a term, so the relevant uncertainty is the price at the next term, not the current one. Hardware generations change faster than terms do, and a commitment to an older accelerator generation can lock in a cost structure that a newer one would beat, which is a risk that no utilization calculation captures. And the load curve is itself affected by price: a lower committed cost may justify serving more traffic, or a higher elastic cost may justify shedding some, so the demand in the model is not independent of the decision being made.

What remains reliable is the break-even condition and the quantile rule. Both are properties of the price ratio and the load distribution, and both can be recomputed as either changes. A capacity decision that states its assumed utilization and its assumed discount can be audited later; one that states only a total cost cannot.