Proxy Boundaries, Forwarded Headers, and the Hop-by-Hop Contract

A reverse proxy is an HTTP endpoint on both sides. It parses a request from the client, then constructs a different request toward the origin, and the two messages are not the same message. Fields that describe the connection rather than the request are consumed at the first hop, the client’s address survives only if the proxy adds it explicitly, and any disagreement between the two parsers about where a message ends is a security boundary rather than a bug.

Every rule in this area follows from that split. The proxy is not forwarding bytes; it is re-serialising a request, and what it chooses to carry across is a policy.

The two classes of field

End-to-end fields describe the request and must survive to the origin: method, target, Content-Type, Authorization, cookies, conditional headers. Hop-by-hop fields describe one connection and must not be forwarded, because their values are false on the next hop.

RFC 9110 lists the standard hop-by-hop fields as Connection, Keep-Alive, Proxy-Authenticate, Proxy-Authorization, TE, Trailer, Transfer-Encoding, and Upgrade. The list is not closed. The Connection field itself names additional fields that are hop-by-hop for that particular message: Connection: X-Internal-Trace means the sender intends X-Internal-Trace to be consumed by the immediate recipient and not passed on. A proxy that strips Connection but forwards the fields it named has done half the job, and the omission is a common way internal topology details leak to clients.

Proxy-Connection is a non-standard field that many clients and older proxies emit, and it has no defined semantics in the current specification. Treating it as equivalent to Connection matches widespread practice; passing it through unchanged does not, because the next hop may apply a different interpretation.

Transfer-Encoding is hop-by-hop for a reason that is easy to miss: the proxy decodes the chunked body and re-encodes it, or terminates the body after decoding. Forwarding the field while forwarding decoded bytes produces a message whose declared framing does not match its content.

Reconstructing the client address

The origin cannot see the client’s address in a proxied request; the peer address is the proxy’s. Something has to carry the original, and the two conventions for doing so are the X-Forwarded-For list, which predates standardisation, and the Forwarded field of RFC 7239.

X-Forwarded-For is a comma-separated list. Each proxy that understands it appends the address of the peer it received the request from, so a request that traversed three hops carries three entries, ordered from the original client on the left to the most recent proxy on the right. Forwarded expresses the same information with named parameters, for, by, proto, and host, and requires quoting around values that contain a colon, which is how an IPv6 address with a port is represented.

The trust problem is identical for both. The leftmost entry is the address the first proxy saw, and the first proxy is the only one that saw the actual client. Any client can send a request with its own X-Forwarded-For header, and a proxy that appends without removing produces a list whose leftmost entry is attacker-chosen. The correct policy is for the outermost proxy to discard the inbound field entirely and start a fresh one, and for every other hop to append. The origin then reads from the right, counting a known number of trusted hops, and ignores everything to the left of the last trusted entry.

A server that simply reads the first element of the list has turned its rate limiter, its access log, and any IP-based access rule into parameters the client controls. Reading the last element is safe only when exactly one proxy is in front; with two, the last element is the first proxy’s address, not the client’s. The number of trusted hops is a deployment fact that has to be configured, not inferred.

var hopByHop = map[string]bool{
    "connection": true, "keep-alive": true, "proxy-authenticate": true,
    "proxy-authorization": true, "te": true, "trailer": true,
    "transfer-encoding": true, "upgrade": true,
}

func toOrigin(h http.Header) http.Header {
    out := http.Header{}
    nominated := map[string]bool{}
    for _, v := range h.Values("Connection") {
        for _, tok := range strings.Split(v, ",") {
            nominated[strings.ToLower(strings.TrimSpace(tok))] = true
        }
    }
    for k, vs := range h {
        lk := strings.ToLower(k)
        if hopByHop[lk] || nominated[lk] {
            continue // consumed at this hop or explicitly single-hop
        }
        out[k] = vs
    }
    return out
}

Rebuilding scheme, host, and path

TLS usually terminates at the proxy, so the origin sees a plain connection and has to be told that the client’s connection was secure. The conventions are X-Forwarded-Proto and Forwarded: proto=https. Without one of them, an application that generates absolute URLs emits http links, and a redirect to a login page becomes a downgrade; an application that sets a session cookie marked secure based on the request scheme may instead fail to set it.

The Host field is a separate decision. A reverse proxy commonly rewrites it to the origin’s internal name or leaves the client’s value in place, and the choice determines whether virtual hosting at the origin works and whether absolute redirects point at the public host. Either way, the value the application trusts for building links should be the one the proxy asserts, not the one the client sent, because the client’s Host is attacker-controlled and is the standard vector for cache poisoning and password-reset link injection.

Path handling adds its own seam. Proxies normalise .. segments and percent-encoding to differing degrees, and the origin applies its own normalisation. If the proxy decodes a path before matching its routing rules and the origin decodes again after matching, a request can match one route at the proxy and a different route at the origin. The safest arrangement is for the proxy to forward the path unchanged and for the origin to normalise exactly once, before matching, as route matching describes.

Via is the field that records the intermediaries a request passed through, and it is advisory. It is useful for diagnosis and useless as a security control, since a client can forge it and a proxy can omit it.

Framing disagreement between hops

The smuggling class of attacks exists because two HTTP parsers can look at the same bytes and disagree about message boundaries. The canonical pair is a request carrying both Content-Length and Transfer-Encoding. A front end that prefers Content-Length forwards a body of the declared length plus whatever follows it; a back end that prefers Transfer-Encoding reads the chunked body, ends it early, and treats the remainder as the beginning of another request. The reverse arrangement, where the front end uses chunked framing and the back end uses the length, produces the same effect in the other direction.

Version downgrades add a third case. A front end speaking HTTP/2 receives the message as a set of fields, where the length has an unambiguous representation, and then converts it to HTTP/1.1 for the origin. If it carries a client-supplied Content-Length across that conversion alongside a body whose length it derived from frames, the origin sees two sources of truth again, and it may pick the wrong one.

The defences are procedural and unglamorous. Every hop must reject messages with ambiguous framing rather than resolve them, which is what HTTP/1.1 message framing argues from the specification’s allowance to treat the combination as an error. Every hop must apply the same strictness to malformed syntax: whitespace before a colon, a duplicate header with differing values, an obsolete folded line, a chunk size with a sign. A front end that is lenient rejects nothing, and the difference between its leniency and the origin’s strictness is the attack surface.

The seam also has to be consistent about connection reuse. An origin connection may only be returned to the pool if the response body was read to completion; a proxy that abandons a partially read response and reuses the socket leaves the remainder in the stream, where it will be read as the beginning of the next response. That is the response-side version of the same boundary confusion, and it requires the proxy to either drain the body or close the connection.

Timeouts and the status they become

The proxy owns two sets of timeouts: how long it waits for the client, and how long it waits for the origin. Both matter, and they map to different status codes that clients and operators read differently.

An origin that fails to connect or returns a malformed response produces 502 Bad Gateway. An origin that accepts the connection but does not answer within the proxy’s deadline produces 504 Gateway Timeout. The proxy’s own refusal to serve, because its worker or connection budget is exhausted, is 503 Service Unavailable, optionally with Retry-After. Collapsing all three into a single code hides whether the fault is the origin’s, the network’s, or the proxy’s, and that distinction is what an operator needs first.

Retrying is where the proxy’s behaviour becomes visible in application semantics. A request that failed before any byte reached the origin can be retried safely; one that failed after the origin received it cannot, unless the method is idempotent or the request carries an idempotency key. A proxy configured to retry on connection reset without that distinction duplicates effects in exactly the cases where the origin was slowest, which is the state the retry was meant to survive. The connection-level view of this problem, and the deadline arithmetic that goes with it, is set out in graceful shutdown and connection draining.