Spot Capacity and the Price of Interruption

Spare capacity is sold at a discount because it is spare: the provider has already paid for the hardware and would rather earn something from it than nothing, on the condition that the buyer accepts eviction when the capacity is needed elsewhere. The discount is therefore not a promotion but a price for a specific risk, and the size of the risk is a property of the workload rather than of the provider. A batch job that can resume from a checkpoint and a stateful service that cannot are offered the same discount and should not accept the same terms.

The discount compensates for a termination notice, not for downtime

The mechanism is that the provider selects instances for reclamation based on its own capacity needs and gives the buyer a short notice period before terminating them. The notice period is a fixed, documented interval, commonly a couple of minutes, and it is the entire compensation for the risk. What the buyer receives is not protection from interruption but warning of it.

The notice period sets the ceiling on what a workload can do in response. Two minutes is enough to checkpoint a progress marker, flush a buffer, deregister from a load balancer, and hand work to another instance. It is not enough to migrate a large in-memory working set, complete a long transaction, or drain a queue of pending requests. The workload’s recovery procedure must therefore be designed to complete within the notice period, and any procedure that cannot is a procedure that will be interrupted in the middle.

Interruption frequency is not a single number. It varies by instance family, by zone, by time of day, and by how much spare capacity exists, and the provider does not guarantee a rate. A cost model that assumes a fixed interruption probability is therefore modelling a quantity the provider explicitly leaves variable. What can be stated is the shape: interruption probability rises with the discount, rises when capacity is scarce, and is not uniform across a fleet at any moment, so a fleet of many instances experiences interruptions continuously even when any individual instance is usually stable.

Reclaiming lost work costs time, and time is the unit being bought

The economics of interruptible capacity are usually stated as a discount against the on-demand rate, and that framing hides the real trade. If a task is interrupted, the work since its last checkpoint is lost and must be repeated. The effective cost per unit of completed work is the price of the capacity divided by the fraction of time that produces durable output.

Let a task run for an expected duration and checkpoint every interval c. On interruption, the expected lost work is c/2 on average, since the interruption lands uniformly within the interval. If interruptions arrive at rate r per unit time, the fraction of capacity spent on lost work is r times the expected loss, and the effective price is the nominal price divided by one minus that fraction:

def effective_price(nominal, interrupt_rate, checkpoint_interval):
    """Fraction of paid time that produces durable output."""
    lost_fraction = interrupt_rate * checkpoint_interval / 2.0
    if lost_fraction >= 1.0:
        return float("inf")            # no progress is ever durable
    return nominal / (1.0 - lost_fraction)

Two conclusions follow directly. First, there is a maximum checkpoint interval beyond which the workload makes no net progress: the denominator reaches zero at c = 2/r, so an interval longer than twice the mean time between interruptions loses as much work as it produces. Second, shortening the checkpoint interval reduces lost work but increases the overhead of writing checkpoints, so the optimum is an interior point rather than the smallest possible interval. If a checkpoint costs a fixed time t to write and the interval is c, the overhead per unit of progress is t/c, and the total cost factor is:

$$\frac{1}{1 - r c / 2} \cdot \left(1 + \frac{t}{c}\right)$$

Minimizing over c places the optimum where the marginal saving from a shorter interval equals the marginal checkpoint cost. For small r, the loss term is approximately linear in c and the overhead term is inversely proportional, so the optimum interval scales as the square root of t divided by r. This is the same trade as any periodic-durability scheme: more frequent writes cost throughput, less frequent writes cost recovery time.

The important consequence for procurement is that the discount is only realized after the checkpointing machinery exists. A workload that has no checkpoint mechanism has an effective price of infinity under interruption, regardless of the discount, because it cannot recover at all. The engineering cost of adding checkpointing is paid once; the discount accrues for as long as the workload runs.

Not all of a workload should be interruptible

A service has parts with different recovery properties, and the discount applies per instance. The stateless tier that handles requests can be replaced in seconds, provided requests are retried, and it is the natural candidate for interruptible capacity. The stateful tier that holds a database or a message broker cannot be replaced without a failover, and a failover has its own cost in availability and in the time to restore service.

Between those extremes sit workloads whose recovery is possible but expensive: a long-running analytics query that must restart from the beginning, a video transcode that must resume from a segment boundary, a machine-learning training job that must reload a model state. Each has a checkpoint cost and a lost-work cost, and each can be evaluated with the same arithmetic. The decision is per workload, not per fleet, and a programme that moves the entire estate to interruptible capacity at once will discover the expensive cases through failures rather than through analysis.

A useful structural pattern is to separate the durable state from the compute. If the compute tier is stateless and the state lives in a service that is provisioned on-demand, then interruption removes compute and costs only the repeated work, while the state remains available. The pattern has a cost of its own: every state access becomes a network call, and the network path between the compute and the state store is metered, as described in egress boundaries and billable data transfer. The traffic is usually small relative to the compute saving, but it is not zero and it grows with how often the compute tier touches the state.

Diversification reduces the variance, not the mean

Interruption is correlated within a capacity pool: when the provider needs capacity in a zone, it reclaims many instances in that zone at once. A fleet concentrated in one pool therefore experiences interruptions in bursts, and the burst pattern is what defeats recovery mechanisms sized for the average rate.

Spreading a fleet across several instance families and zones reduces the correlation, so the fraction of the fleet interrupted at any moment is closer to the average than to the worst case. The cost of spreading is that each pool has its own price and its own availability, so a diversified fleet has a blended price above the cheapest pool’s. The trade is between the cheapest nominal price and the stability of the effective price, and the correct point depends on how much of the workload’s capacity can be lost simultaneously without violating its service objective.

Capacity reservations interact awkwardly with this. A reservation of interruptible capacity in one pool concentrates risk in that pool; a reservation across pools multiplies the fixed cost of holding capacity. The alternative is to hold on-demand capacity as a floor and use interruptible capacity above it, which caps the loss at the interruptible portion and prices the floor at the on-demand rate. The floor’s size is set by the same headroom logic as any autoscaling policy, covered in autoscaling policies and the cost of headroom, with the difference that the reaction time must accommodate an interruption notice rather than a load change.

Where the discount stops being a saving

Three conditions turn interruptible capacity into a net loss. The first is a workload whose recovery cost exceeds the discount: a job that must restart from zero after every interruption and whose interruptions arrive more often than its runtime will never finish, and the capacity it consumes produces nothing. The second is a workload whose interruption causes a commitment to go unconsumed: if a reservation covers the on-demand rate for a family and the workload moves to interruptible capacity in a different pool, the reservation’s discount is lost while the interruptible discount is gained, and the two do not necessarily net out.

The third is subtler. Interruption changes the fleet size over time, and a fleet that shrinks unpredictably has a lower average utilization of any capacity that is held for it. Headroom must cover the interruption notice period in addition to the load, so a fleet that runs close to its target utilization on interruptible capacity will breach its latency objective whenever an interruption occurs, not because the load changed but because capacity disappeared. The headroom that prevents this is priced at the on-demand rate if it is held as a floor, which is the honest accounting: the discount on the interruptible portion is partly paid back by the on-demand capacity that must exist to cover its absence.