Standing Queues and the Buffer Sizing Trade-Off

A queue that never drains is not doing what a queue is for. Buffers exist to absorb bursts so that a link stays busy when traffic arrives unevenly; a buffer that stays full has stopped absorbing anything and is instead adding a fixed delay to every packet that crosses it. The condition is easy to create accidentally, because the transport protocol that fills the queue is designed to keep increasing its sending rate until something signals it to stop, and a large buffer provides no signal until the buffer is nearly full.

The term for this is bufferbloat, and the important property is that it is not congestion in the usual sense. The link is not oversubscribed on average; there is no sustained loss; throughput is what the link rate allows. The damage is entirely in delay, and it persists for as long as the sending pattern persists.

How a standing queue forms

Consider one sender with a large congestion window and one bottleneck link. The sender’s rate exceeds the link rate, so the queue grows. When the queue approaches full, the sender eventually loses a packet or receives a congestion signal and reduces its rate below the link rate, and the queue drains. The cycle repeats.

The trouble is what happens at the bottom of that cycle. A sender that reduces its rate and then immediately probes upward again spends most of its time at or above the link rate, so the queue spends most of its time non-empty. The queue depth settles at a level where the sender’s average rate equals the link rate, and that level is typically a substantial fraction of the buffer. Every packet crossing the link then waits behind that standing backlog, and the added delay is the backlog divided by the link rate.

The arithmetic is direct. A 20 Mbit/s access link with a 1,000-packet buffer, each packet 1,500 bytes, holds 12 megabits of data when full. If the standing queue sits at half the buffer, that is 6 megabits, which at 20 Mbit/s takes 300 milliseconds to drain. A voice or game packet arriving at that moment waits behind all of it. The link is delivering its full rated throughput and the path is unusable for anything interactive.

Deep buffers appear where the cost of memory is low and the cost of a dropped packet is perceived as high: consumer routers, home gateways, cable modems, and switch ports built with large shared memory pools. A device with a 100-millisecond buffer at its rated rate is not misconfigured by its own standards; it is configured to maximise throughput for bulk transfer, at the cost of latency for everything sharing the link.

The bandwidth-delay product and the rule of thumb

The classical sizing rule comes from the observation that a TCP connection’s throughput is limited by the window it can keep in flight. To fill a link of rate $C$ over a round-trip time $R$, the sender needs a window of $C \times R$ bits in flight, the bandwidth-delay product. A router on that path must be able to hold at least one bandwidth-delay product of data if it is to avoid dropping packets that a sender legitimately has in flight, because at the moment a link becomes free, a full window of packets may be arriving.

For a 1 Gbit/s path with a 100 ms round trip, that product is 100 megabits, or about 8,300 full-size packets, roughly 12 MB. Buffers of that size are the origin of the intuition that more buffer is better, and on a backbone link with a small number of high-rate flows the reasoning holds.

The rule breaks down when the number of flows is large. Appenzeller, Keslassy, and McKeown argued that a router with $N$ flows needs a buffer of roughly $C \times R / \sqrt{N}$ rather than $C \times R$, because independent flows’ windows do not all peak simultaneously; the statistical multiplexing among them reduces the peak backlog. On a core link carrying tens of thousands of flows this suggests buffers far smaller than a full bandwidth-delay product, and the argument is the basis for much smaller core buffers than the classical rule implies.

Neither rule applies well to an access link carrying one or two flows. There, $N$ is small, the statistical argument gives nothing, and the buffer needed to keep the link busy is a genuine bandwidth-delay product. But the buffer that keeps the link busy is also the buffer that produces the standing queue, because with one flow there is no other flow to fill the gap when the single sender backs off. The device ends up either wasting capacity or adding hundreds of milliseconds of delay, and the choice is made by buffer size alone.

Recognising the condition from measurements

The signature of bufferbloat in a measurement is a delay that rises with load and stays high, rather than spiking briefly. A round-trip measurement taken while a bulk transfer runs shows a stable elevated value, sometimes for minutes, and drops back to the idle value within a few round trips of the transfer finishing. Ordinary transient congestion produces spiky delay that returns to baseline between bursts.

The second signature is that the elevated delay is close to the buffer’s drain time. If the idle round-trip time is 20 ms and the loaded round-trip time is 320 ms, the extra 300 ms is a queue of 300 ms at the bottleneck rate. Dividing the extra delay by the bottleneck rate gives the standing backlog in bytes, which can be compared with the device’s buffer specification to confirm the diagnosis.

A third signature is that loss stays near zero while delay is high. When a link is genuinely oversubscribed, the buffer overflows and packets are dropped, and the transport reacts by reducing its rate. When a standing queue merely sits in a large buffer, nothing overflows, nothing signals the sender, and the delay persists indefinitely. This is why a network can be simultaneously “not congested” by a loss-based dashboard and unusable by a latency-based one.

The measurement itself has to be done carefully. A probe sent from the same host that is running the bulk transfer competes with it and measures a different queue than an idle observer would. A probe sent over a different path measures a different path. The useful technique is to measure delay to a target on the far side of the suspected bottleneck while loading that bottleneck deliberately, which is what the delay-under-load methodology in measuring latency under load describes.

Control, not capacity, is the lever

The fix for a standing queue is not more bandwidth. Adding capacity raises the drain rate and shortens the queue’s drain time, but the sender’s congestion control will simply fill the larger pipe until the queue re-forms, and the delay returns. Capacity changes the scale of the problem without changing its existence.

The fixes that work operate on when packets are dropped or marked. A queue management algorithm that starts signalling before the buffer is full gives the sender a chance to reduce its rate while the queue is still short. Random early detection, described in RFC 2309, drops or marks packets with a probability that rises with the average queue length, so a sender sees an early signal and the queue is kept away from the full mark. Modern variants use the queue’s sojourn time rather than its length as the signal, which stays correct when the link rate varies.

The complementary fix is at the sender. A congestion controller that treats delay as a signal, rather than waiting for loss, backs off before the queue grows large. The interaction between such a controller and the queue is the reason a queue management algorithm and a delay-sensitive sender have to be designed together rather than independently, and the queue-side half of that design is set out in active queue management and latency under load.

A third fix is at the scheduler. Giving interactive traffic its own queue, so that it does not sit behind bulk traffic, removes the delay for that class even when the bulk queue is deep. This is the practical answer on links where the sender cannot be changed, and it costs a scheduler configuration rather than a protocol change.

What buffer size is actually for

A buffer’s legitimate job is to cover the interval during which the output link is busy with a packet that has already been scheduled while new arrivals continue. Without any buffer, a link would idle whenever an arrival happened to coincide with a transmission in progress, and utilisation would drop. The minimum useful buffer is therefore about one maximum-size packet, and the useful range extends to the amount needed to cover the round-trip dynamics of the flows sharing the link.

Above that range, additional buffer buys throughput for bulk flows and costs delay for everything else. Where the balance sits depends on how many flows share the link and what they are. A core link with ten thousand flows and no interactive traffic can use large buffers productively. A home gateway with three flows and a video call cannot, and its buffer should be small or actively managed. Treating both as the same sizing problem is the mistake that produces bufferbloat, and it is a mistake made by a default configuration rather than by any deliberate decision.