The Effective Window: Congestion Window, Receive Window, and Buffers

A TCP sender is subject to three limits that are computed by different parts of the system, updated on different events, and expressed in the same unit. The receive window comes from the peer and represents buffer space. The congestion window is maintained locally and represents an estimate of network capacity. The send buffer is a local allocation and represents memory the application is allowed to occupy.

The sender transmits when all three permit it. The minimum is what governs, and which of the three is the minimum determines what an application can do about a slow connection.

The congestion window is an inference, not a measurement

The receive window is a fact: the peer has this much buffer free. The congestion window is not. No component of the network reports capacity to the sender. Routers do not send “I can handle 12 Mbit/s on this path” messages in TCP; they drop packets when their queues overflow, and the sender infers congestion from the drops.

That inference shapes the whole mechanism. Because the signal is binary and delayed by a round trip, the sender must probe: increase the window until something breaks, then back off. The standard algorithm, described in RFC 5681, has two phases with different growth rules.

Slow start doubles the window per round trip. The name is misleading: the window starts small, at a few segments, but its growth is exponential. For each acknowledgement that reports new data, the window grows by at most one segment, so a window of w segments produces w acknowledgements in a round trip and the window becomes 2w. The initial window in RFC 5681 is two, three, or four segments depending on the maximum segment size; the value of ten segments in common use comes from an experimental specification that raised it. From ten segments, a thousand is about seven doublings, and so about seven round trips.

Congestion avoidance grows the window by one segment per round trip. Each acknowledgement adds MSS × MSS / cwnd bytes to the window, so a full window of acknowledgements adds exactly one segment. The growth is linear, and the rate of increase is deliberately slow: one segment per round trip raises the sending rate by MSS / RTT, which on a 100 ms path is about 14 KB/s per round trip for a 1,460-byte segment, so on the order of a hundred kilobytes per second per second.

The switch between the two happens at the slow start threshold, ssthresh. Below it, exponential; above it, linear. The threshold is set when a loss is detected: to half the amount of data that was in flight at the moment of detection, with a floor of two segments. The value is derived from the flight size rather than from the window variable, because the window may have been increased after the lost segment was sent, and the sender wants the estimate of what the path was actually carrying.

/* Window update on each acknowledgement, simplified: no delayed ACK, no SACK. */
struct cc_state {
    uint32_t cwnd;        /* bytes */
    uint32_t ssthresh;    /* bytes */
    uint32_t mss;
    uint32_t credit;      /* congestion-avoidance accumulator, in bytes */
};

void on_ack(struct cc_state *s, uint32_t newly_acked)
{
    if (s->cwnd < s->ssthresh) {
        s->cwnd += newly_acked;                 /* slow start: one byte per byte acked */
    } else {
        /* congestion avoidance: accumulate mss*mss/cwnd per acked byte */
        s->credit += s->mss * newly_acked;
        while (s->credit >= s->cwnd) {
            s->credit -= s->cwnd;
            s->cwnd += s->mss;
        }
    }
}

void on_loss(struct cc_state *s, uint32_t flight)
{
    s->ssthresh = flight / 2;                   /* halve what was actually in flight */
    if (s->ssthresh < 2 * s->mss) s->ssthresh = 2 * s->mss;
    s->cwnd = s->mss;                           /* timeout: restart from one segment */
}

The counting detail matters more than it appears. Adding one segment per acknowledgement is correct only when each acknowledgement covers one segment. A receiver that delays acknowledgements, covering two segments each, halves the apparent number of acknowledgements and therefore halves the growth rate. Taking the acknowledged byte count as the input fixes the halving but introduces the opposite problem: an acknowledgement that covers a large burst, which can happen when a stalled receiver drains its buffer at once, credits the window with the whole burst in one step and produces exactly the bursty transmission that slow start is meant to avoid. RFC 3465 settles on a middle course, crediting at most a fixed amount per acknowledgement, and that limit is why the code above takes newly_acked as an argument rather than incrementing by a constant.

Why the window collapses rather than shrinking

A loss response does not reduce the window to the point of loss. It halves it. The reason is that the loss occurred at some point after the window had grown past what the path could carry, and the sender does not know by how much. Halving is a bisection step: it returns to a window presumed to be below the failure point, at the price of discarding up to half the available capacity.

The alternative — returning to the window at which the previous loss occurred — rests on a historical estimate, and the network moves. A path that supported 100 segments a minute ago may support 80 now, so the remembered value is not a bound on the present. Halving measures the current condition, in that it divides what was actually in flight at the moment of detection.

Timeout-driven loss is treated more severely than duplicate-acknowledgement-driven loss. A timeout resets the window to one segment and re-enters slow start; three duplicate acknowledgements halve the window and continue in congestion avoidance. The distinction is evidential. Duplicate acknowledgements mean later segments are arriving, so the path is delivering data and the loss is isolated. A timeout means nothing has arrived for a full retransmission interval, which is consistent with a path that has stopped delivering entirely. The stronger response is proportionate to the weaker evidence.

This asymmetry is also why spurious timeouts are expensive. A timeout caused by a delayed acknowledgement, not by loss, resets the window to one segment and forces the connection to re-probe capacity from scratch. The recovery can take many round trips. The mechanisms that detect the spurious case are covered in Timestamps, RTTM, and Protection Against Wrapped Sequences.

Which limit binds

The three limits produce four regimes, and each has a different remedy.

When the congestion window binds, the connection is limited by its estimate of the network. The path may have more capacity, but the sender has not probed far enough to find it, or it has recently backed off. Remedies are on the transport: a better congestion control algorithm, a larger initial window, or a mechanism that distinguishes congestion loss from corruption loss.

When the receive window binds, the peer’s buffer is the constraint. The remedy is on the peer: a larger receive buffer, or window scaling to express it, as described in Advertised Windows and Zero-Window Probing. A sender cannot fix this, though it can avoid making it worse by not filling the window with data that the application will discard.

When the send buffer binds, the local allocation is the constraint. The remedy is local: a larger SO_SNDBUF, or an application that writes in larger units. This regime is common and frequently misdiagnosed as a network problem, because the symptom, a write that blocks or returns a short count, looks like backpressure from the network.

When none binds, the connection is running at the rate the application supplies data, and the transport is not the bottleneck.

The effective limit is therefore

inflight_max = min(cwnd, SND.WND, SO_SNDBUF - unsent)

and the throughput ceiling implied by it is roughly inflight_max / RTT. The expression is an approximation, because acknowledgements arrive continuously rather than in round-trip batches, but it is the right first-order model and it correctly predicts that a connection with a large window on a long path is limited by the window while the same window on a short path is not.

The bandwidth-delay product as the target

The quantity that a window must reach to fill a path is the bandwidth-delay product: the number of bytes that fit in the pipe at the path’s capacity and round-trip time. A 1 Gbit/s path with a 50 ms round trip holds about 6.25 MB in flight. A window below that value leaves capacity idle; a window far above it builds a standing queue.

The receive window and the congestion window are both trying to reach this value, from different directions and with different information. The receiver’s window is set by buffer allocation and can be made arbitrarily large if memory permits. The congestion window converges to the bandwidth-delay product only through probing, and it converges to slightly more than the product, because the loss that stops its growth occurs after a queue has formed.

This is the origin of the bufferbloat problem: a path whose buffers are much larger than the bandwidth-delay product lets the congestion window grow far past the product before any loss occurs, and the excess sits in router queues as latency. The window is “working” in the sense that throughput is high, while the actual measure that matters to interactive traffic, queueing delay, has grown by the buffer size divided by the link rate.

Measuring the limit from outside

An application cannot read the congestion window directly on most systems; the value is kernel state and is not exposed through a portable interface. It can be inferred, roughly, from the amount of data in flight observed at the socket: if a write of n bytes completes without blocking and the peer’s acknowledgements keep pace, the effective window is at least n.

The more reliable approach is to measure the constraint by its effect. A connection whose throughput is stable and whose round-trip time is flat is not queue-limited. A connection whose throughput is stable and whose round-trip time grows with load is queue-limited, and the growth is the queue occupancy. Throughput alone does not separate the three cases; round-trip time, the peer’s advertised window, and the local socket buffer size each have to be examined, because all three limits produce the same symptom at the socket.