Persistent Connections, Pipelining, and Connection Reuse

In HTTP/1.0, every request opened a connection and closed it after the response. For a page with twenty embedded resources, that meant twenty TCP handshakes, each costing a round trip before any HTTP bytes moved, plus a slow-start ramp for each new connection. HTTP/1.1 made persistence the default: the connection stays open and the next request reuses it. The change is small in the specification and large in its consequences, because a shared connection introduces ordering, lifetime, and ownership questions that a per-request connection never had.

How a connection ends

A persistent connection is the default in HTTP/1.1. The client does not need to ask for it, and the server does not need to advertise it. To close, either side sends Connection: close, and that field applies to the message it appears in: a server sending it in a response promises to close after that response, and a client sending it in a request promises to close after receiving the response.

A response with Connection: close still has to be read to its end. The close is the end-of-message signal only when the response has no length of its own; a response that declares Content-Length or chunked framing ends at that boundary, and the connection then closes. RFC 9112 requires a sender to close the connection after sending a message whose body is delimited by closure, and requires a recipient to treat the close as the end of that body. That is the fallback path: a response with no Content-Length, no Transfer-Encoding, and a status that permits a body is delimited by the connection ending. It works, but the connection cannot be reused afterwards: the recipient learns where the body ends only when the connection drops.

HTTP/1.0 clients had the inverse default. A request without Connection: keep-alive closed after the response, and a server that wanted persistence had to send the field back. A server that receives an HTTP/1.0 request with Connection: keep-alive may keep the connection open, but the response must still be self-delimiting, since a 1.0 client may not implement chunked transfer encoding.

Idle timeouts are where persistence meets operations. A server cannot keep connections open forever; each idle connection consumes a file descriptor and some memory. RFC 9112 places no requirement on the length or existence of an idle timeout, so implementations choose their own. The client, meanwhile, may believe the connection is still usable and send a request into a socket the server has already closed. The result is a request that fails after a full round trip, or a response that never comes. The client cannot distinguish a connection that died from one that is merely slow, and this ambiguity is why connection pools either check an idle socket before use or retry a request that fails before any response byte arrives, rather than trusting a single reused socket.

Pipelining and why it failed

Pipelining is the technique of sending several requests on one connection without waiting for each response, then reading the responses in order. The specification permits it, and imposes one firm requirement on the receiving server: responses must be returned in the same order as the requests, so a server may not interleave them. It does not require the server to read ahead.

Ordering is the constraint that killed it. Because responses must come back in request order, a slow first request blocks every request behind it. The client gains nothing on latency for the blocked requests and has given up the ability to cancel the first one cleanly. Worse, the client must be able to match responses to requests without ambiguity, and if any request is one the server might answer with a 1xx interim response, the mapping becomes harder to track.

Servers also have discretion about how much pipelining to tolerate. A server may read one request at a time from the socket and process it before reading the next, which is conformant and defeats the purpose. It may buffer requests, which introduces memory pressure and a way to amplify a small connection into a large amount of queued work. Because the specification does not require the server to read ahead, a client that pipelines cannot rely on the performance benefit.

The practical outcome is that browser clients do not pipeline at all, and an intermediary that receives a pipeline may forward it as received or deal with the requests one at a time. HTTP/2 removed the constraint at the protocol level, multiplexing frames from several requests on one connection. The contrast is the useful lesson: HTTP/1.1’s ordering constraint is a consequence of framing, not of semantics.

Reuse, pooling, and the stale-socket problem

A client that keeps a pool of connections faces a decision on every request: take an idle connection or open a new one. Taking an idle one risks a socket the server has already closed. Opening a new one costs a handshake. The common resolution is to send the request on the idle connection and, if it fails before any response bytes arrive, retry on a new connection. That retry is safe only for idempotent methods, which is the connection between this subject and method semantics.

The failure mode is worse than a clean error when the request was not idempotent. A POST sent into a closed socket may have been fully received by the server before the close, in which case the server processed it and the client will never see the response. Retrying duplicates the effect; not retrying loses it. Applications that care about this need an idempotency key that the origin can deduplicate, as described in that article.

There is a second, subtler failure: a server may close a connection while the client is mid-request. If the close happens after the server has read the head but before the body arrives, the server should send a 400 or 408 and close, but some servers simply close. The client sees a truncated write or a reset. A client that treats a reset as a definitive rejection will misreport a transient condition as a permanent one.

What a server must do to keep a connection usable

Three obligations, each of which is a place implementations get it wrong.

Every response must be self-delimiting unless the server intends to close. That means either Content-Length or Transfer-Encoding: chunked, and never both in a way that disagrees. A response to HEAD has no body but may carry the length the body would have had; a 204 has no body and no length field is needed. The full set of rules is in message delimitation and request smuggling, and the connection-level consequence is that a response whose length the client cannot determine forces the connection to close.

The server must consume any request body it did not read. A handler that reads the first few bytes of an upload and then responds without draining the rest leaves those bytes in the socket, where the next request’s parser will see them as the start of a request line. The specification’s framing rules make this a correctness requirement, not a courtesy: the request body must be read to its end, discarded, or the connection closed. Implementations handle this by either draining a bounded amount or closing, and the choice affects how much a client can upload before being cut off.

A server that answers before reading the complete request must close the connection afterwards: RFC 9112 requires it to read the entire body or close after sending its response. That licence is what makes early rejection practical — a 413 or 417 can be sent while the client is still uploading, because the alternative is to keep reading a body the server has already decided to reject, which spends bandwidth on both sides for no purpose. The close is mandatory, since the unread body would otherwise be parsed as the next request.

// One request-response cycle on a persistent connection.
for {
    req, err := readRequest(br)
    if err != nil {
        break // client closed, or the message was malformed
    }
    resp := handle(req)
    if err := writeResponse(bw, resp); err != nil {
        break
    }
    if !req.wantsKeepAlive() || resp.forbidsKeepAlive() {
        break
    }
    // The body must be at EOF before the next request is parsed.
    if err := req.drainBody(); err != nil {
        break
    }
}
conn.Close()

The loop has no per-request state beyond the reader and writer, which is what makes persistence cheap. Everything expensive is per connection: the handshake, the TLS session, the buffer allocation.

The cost model that changed

The reason persistence matters is that connection setup is not free. A TCP handshake is one round trip before the first byte of the request can be sent. A TLS handshake adds one or two more, depending on version and whether a session is resumed. On a path with 50 ms of round-trip time, three round trips is 150 ms of latency before the server has seen a single byte of the request. Reusing a connection removes all of it.

That arithmetic explains the shape of browser behaviour: a browser opens several connections to one origin in parallel because a single connection serialises request-response pairs, and parallelism across connections is the only way to overlap them under HTTP/1.1. It also explains why the per-origin connection limit exists and why domain sharding was once a technique: spreading resources across hostnames multiplied the number of connections a browser would open. That technique costs DNS lookups and TLS handshakes per host and is largely obsolete now that multiplexing exists.

The residual case for HTTP/1.1 persistence is the one where multiplexing is unavailable: simple clients, command-line tools, and service-to-service calls with one request at a time. There, a pool of a few connections with an idle timeout and a retry-on-stale-socket rule is the whole design.