Why Queueing Delay Rises Sharply Near Link Utilization

A link carrying half its capacity adds almost nothing to the delay of a packet crossing it. The same link at ninety percent of capacity adds a multiple of the packet’s own transmission time, and at ninety-eight percent the queueing term dwarfs every other contribution the link makes. The transition is not gradual and it is not a hardware defect; it is the shape of the waiting-time function, and it is the reason capacity planning uses utilisation thresholds that look conservative.

The simplest model that reproduces the shape is a single server with a queue, Poisson arrivals, and exponentially distributed service times, usually written M/M/1. Real traffic violates the Poisson assumption badly, but the qualitative behaviour of the model survives every relaxation, which is why it remains the reference picture.

The waiting-time formula and what each symbol means

For M/M/1, with arrival rate $\lambda$ packets per second and service rate $\mu$, the utilisation is $\rho = \lambda/\mu$, and the mean number of packets in the system is

$$N = \frac{\rho}{1-\rho}$$

By Little’s law, the mean time in the system is $N/\lambda$, which gives the mean sojourn time

$$T = \frac{1}{\mu - \lambda} = \frac{1}{\mu}\cdot\frac{1}{1-\rho}$$

The factor $1/\mu$ is the mean serialization time of one packet. Everything else is the queueing multiplier $1/(1-\rho)$. At $\rho = 0.5$ the multiplier is 2, so the mean delay is twice the bare transmission time. At $\rho = 0.8$ it is 5. At $\rho = 0.9$ it is 10. At $\rho = 0.95$ it is 20. At $\rho = 0.98$ it is 50.

The numbers make the knee concrete. Take a 1 Gbit/s link serving packets averaging 1,000 bytes, so $1/\mu$ is 8 microseconds. At half load the mean delay is 16 microseconds, indistinguishable from the serialization term in any real measurement. At 90 percent load it is 80 microseconds. At 98 percent it is 400 microseconds. Nothing about the link changed between those cases except how many packets were offered to it.

The same arithmetic on a congested access uplink is much less forgiving. A 10 Mbit/s uplink with 1,500-byte packets has a serialization time of 1.2 milliseconds. At 90 percent utilisation the mean queueing delay is 10.8 milliseconds, and the sojourn time is exponential with a mean of 12 milliseconds, so its ninety-fifth percentile is about three times that mean and a packet in the tail spends over 30 milliseconds in the system. On a link that rate, the tail is what users perceive, not the mean.

Why the curve bends where it does

The knee has a mechanical explanation. When a packet arrives at a link, the question is whether the server is busy and how many packets are ahead. At low utilisation the server is idle most of the time, so an arriving packet usually finds an empty system and waits zero. At high utilisation the server is busy most of the time, and the number of packets ahead grows because arrivals and departures are only balanced on average, not instantaneously. The variance of the backlog, not its mean, is what produces the delay, and that variance grows without bound as $\rho$ approaches one.

Formally, the queue length distribution of M/M/1 is geometric with parameter $\rho$, so the probability of finding $k$ packets in the system is $(1-\rho)\rho^k$. At $\rho = 0.5$ the chance of finding ten or more packets ahead is below one in a thousand. At $\rho = 0.9$ it is about 35 percent. The tail of the delay distribution, not the mean, is what shows up in a latency percentile, and it degrades faster than the mean does.

Real traffic is burstier than Poisson, which makes the curve steeper. A model with general interarrival and service distributions, commonly quoted as the Pollaczek–Khinchine formula for M/G/1, gives a mean waiting time proportional to $(1 + C_s^2)\rho / (2(1-\rho))$ times the service time, where $C_s^2$ is the squared coefficient of variation of service time. When service times are exponential, $C_s^2 = 1$ and the expression reduces to the M/M/1 result. A link whose service times vary more than that, which a mix of full-size segments and small control packets produces, has $C_s^2 > 1$ and therefore more delay at the same utilisation. Arrival patterns burstier than Poisson fall outside this model altogether, and self-similar traffic, which real aggregates resemble, behaves worse still at high load because bursts arrive at many timescales at once.

Utilisation targets in practice

The shape of the curve explains why planning targets cluster in a narrow band. A target of 50 percent leaves the multiplier at 2 and tolerates growth to 75 percent before the multiplier reaches 4. A target of 70 percent leaves the multiplier near 3.3 and reaches 5 at 80 percent, so a 14 percent increase in offered traffic costs half again as much mean delay. Beyond roughly 80 percent the remaining headroom buys very little additional throughput while the delay multiplier grows quickly, so the marginal capacity is better spent elsewhere.

These are not arbitrary conventions. They are the points at which the derivative of the delay curve becomes uncomfortable: between $\rho = 0.5$ and $\rho = 0.8$ the multiplier rises from 2 to 5, and between 0.8 and 0.95 it rises from 5 to 20. The second interval is where latency-sensitive traffic starts to fail its budget, and it is where measurement noise in utilisation estimates starts to matter, because near $\rho = 0.9$ a three percent error in the estimated arrival rate moves the predicted multiplier by tens of percent.

Aggregate links get more headroom than this reasoning alone would suggest because of the law of large numbers across flows. A 100 Gbit/s backbone link carrying thousands of independent conversations has arrivals that smooth out relative to a single flow, so its delay curve is flatter for the same utilisation. A single 10 Mbit/s access link carrying one video call and one file transfer has almost no aggregation, so its curve is steeper. The same 80 percent target means different things at those two scales.

Where the model misleads

The M/M/1 curve assumes an infinite buffer, a single server, and work-conserving service in arrival order. Each assumption fails somewhere.

With a finite buffer, packets that would have waited are dropped instead. A full buffer converts delay into loss, which is why a link at very high utilisation can show a moderate mean delay and a large loss rate: the long waits were never completed, they were discarded. Any delay measurement taken at high load without a simultaneous loss measurement is incomplete, and the tail-drop behaviour that produces the conversion is treated in active queue management and latency under load.

With a single FIFO queue, all traffic shares one delay distribution. A link that mixes bulk transfers with interactive requests gives the interactive traffic the same tail as the bulk traffic, which is the motivation for priority and fair queueing rather than for more capacity. Separate queues change the curve for a class of traffic without changing utilisation, and the design of those queues is covered in priority and fair queueing for mixed traffic.

With exponential service times, the model predicts a smooth tail. Real packet sizes are bimodal, clustered near the maximum segment size and near small control packets, so the actual delay distribution has steps rather than a smooth exponential decay. A measurement that reports only a mean will miss the step.

One further caveat concerns what the utilisation figure means. Link utilisation measured over a five-minute interval averages away the bursts that produce the queueing, so a link reported at 40 percent average utilisation can be saturated for tens of milliseconds at a time. The queue does not see the average; it sees the instantaneous arrival pattern, and the mismatch between the two is why average-utilisation dashboards and user-perceived latency disagree so often.