Middlebox Ossification and Frozen Invariants

A protocol specification describes the behaviour of two endpoints. The network between them contains devices that were not part of the design: firewalls, load balancers, intrusion detection systems, transparent proxies, and carrier-grade NAT. Those devices have to make decisions about traffic, and to make decisions they parse. Once a device parses a field, that field’s current encoding becomes a dependency, and the dependency survives long after the endpoints have moved on.

This is ossification in the precise sense: the protocol can no longer change a visible part of its encoding, not because the endpoints cannot implement the change, but because the path will drop the traffic. The constraint is not in the specification and it is not in either implementation. It is distributed across the deployed base of devices that nobody owns and nobody can update.

The mechanism: a parser that must be right

A middlebox that inspects TLS ClientHello messages does so to extract the server name, so that it can apply a policy without decrypting the connection. To extract the server name it must find the extension of the right type. To find the extension, it must walk the extension list. To walk the list, it must understand the outer length fields, and it must decide what to do when it encounters a type it does not recognise.

If it decides to reject the message, then any new extension becomes a connectivity failure. If it decides to skip, then the message passes and the extension is harmless. The device’s author makes that decision once, in code that is then frozen into firmware and deployed to thousands of sites. The correct choice is skip, and the reason implementations get it wrong is that skipping requires the length to be validated, and validation is where the bugs are, so the easier implementation is to reject anything unexpected.

The same pattern appears at every layer. An IPv4 router that parses options to implement source routing will drop packets with option types it does not know, if the option is marked as requiring understanding. An HTTP proxy that parses request headers to apply content filtering will mishandle a header with an unusual syntax. A NAT that parses TCP to track connection state will mis-track a connection whose handshake includes options it did not anticipate.

The ossification is not caused by malice or incompetence. It is caused by the correct engineering decision, made locally, that an unknown input is a risk. Every device author makes that decision, and the aggregate is a network that rejects novelty.

Why encryption is the structural answer

The most effective response is to make the fields unreadable to the path. If a middlebox cannot parse a field, it cannot depend on it, and the field remains free to change.

TLS 1.3 applied this wholesale. In TLS 1.2, the entire handshake was in the clear, so every extension, every cipher suite list, and every certificate was visible. In TLS 1.3, everything after the ServerHello is encrypted under handshake keys, which means a middlebox sees the ClientHello and the ServerHello and nothing else. The set of fields that can ossify is reduced to those two messages.

The reduction has a cost. Network operators who relied on reading the certificate to apply policy lost that ability, and the resulting argument over encrypted client hellos, enterprise interception, and middlebox policy was substantial. The technical point is that the operators’ need was real and the protocol’s need was also real, and the resolution was to encrypt the part that mattered for extensibility while leaving the server name visible in the ClientHello, in the clear, for the devices that need it.

QUIC went further by encrypting almost everything, including the transport parameters that would otherwise be visible. The design decision there was that the handshake is short and the path does not need to see it; the parts that the path does need, such as the connection ID, are deliberately visible and deliberately fixed in format. That is the shape of the compromise: a small set of fields is designated as path-visible and frozen, and everything else is encrypted and free.

The invariants a path device actually needs

The fields that remain visible are chosen because a path device needs them to do its job. For a router, the invariant is the IP header: the version, the source and destination addresses, and the length. For a NAT, the invariant is the five-tuple and the transport ports. For a firewall, the invariant is enough of the transport header to identify the flow.

The design question is how much of each invariant to expose. Too little and the device cannot function, so it will either fail or find a heuristic. Too much and the exposed fields ossify. The heuristic case is the dangerous one, because a device that cannot parse the field it needs will guess from other signals, and the guess becomes a dependency on those signals.

An example is the practice of inferring the application protocol from the port number. A firewall that allows port 443 because it expects TLS will, if it cannot parse the handshake, still allow the port and inspect nothing. That is a policy hole, and it also means the port number becomes load-bearing: an application that moves to a different port loses connectivity, not because the protocol changed but because the policy was written against the port.

The general principle is that any field a path device uses becomes part of the protocol’s effective contract, whether or not the specification says so. The contract can be narrowed by encrypting the field, or widened by documenting it, and the choice is a design decision with a real cost on both sides.

Ossification by measurement, not just by policy

A subtler form of ossification comes from measurement rather than from enforcement. If a monitoring system records that all observed ClientHello messages have a particular structure, then a new structure will appear as an anomaly, and anomaly detection will flag it. The flag may not block the traffic, but it produces an operational cost: support tickets, investigations, and eventually a policy that suppresses the anomaly by blocking the traffic.

This is why protocol designers care about the distribution of observed messages, not just their legality. A new extension that is legal but rare will be flagged by any system that has learned the common shape. GREASE addresses this by making unusual values common, so that the anomaly detector’s baseline includes them. Randomised padding does the same for message sizes.

The measurement problem has a feedback loop. A monitoring system trained on traffic that lacks an extension will treat the extension as anomalous; the anomaly leads to blocking; the blocking means the extension stays rare; the rarity confirms the baseline. Breaking the loop requires either making the extension common enough to be normal, or making the monitoring unable to see it.

What a protocol can do about a device it cannot update

There are four structural strategies, and each has a cost.

Encrypt the field. The device cannot parse it, so it cannot depend on it. The cost is that legitimate uses of the field by the path are lost, and the loss is usually discovered by an operator whose policy stops working.

Change the encoding so that the old parse fails safe. If a new field is placed where an old parser expects a length, the old parser will compute a wrong length and reject the message rather than misparse it. That is a deliberate incompatibility, and it only helps if the rejection is visible to the sender as a protocol error rather than as a silent drop.

Provide a fallback that the path can see. If a new feature has an old-feature encoding that is semantically equivalent, the path can be given the old encoding and the endpoints can negotiate the new one out of band. This is the pattern behind the visible-invariant design: the path sees a version it knows, and the endpoints speak something else.

Accept the freeze. Some fields will ossify and the protocol will work around them. This is the honest option when the field is genuinely needed by the path and the cost of encrypting it is too high. The workaround is usually to add a new field elsewhere rather than to change the frozen one, which is why protocols accumulate parallel mechanisms.

The choice among these is not purely technical. It depends on who operates the devices, whether they can be updated, and what the regulatory environment allows. A protocol designed for a network with no middleboxes can ignore the problem; a protocol designed for the public internet cannot. It also depends on how much of the message the endpoints can afford to make opaque, which is why the visible-invariant discipline is a design constraint rather than an implementation detail, and why tolerant parsing of the fields that remain visible is so consequential.

The record of what actually happened

TLS 1.3 is the clearest documented case, because the specification states its reasoning rather than leaving it implicit. The record layer fixes the version bytes at 0x0303 for every protected record regardless of the negotiated version, so a device that has learned to expect TLS 1.2 framing continues to see it. The handshake after the ServerHello is encrypted, so the certificate, the extensions the server chose, and the finished messages are not visible to the path. A compatibility mode exists in which the endpoints emit a dummy ChangeCipherSpec record at the points where a TLS 1.2 handshake would have one, so that devices tracking handshake progress by counting those records do not stall.

Each of those is a deliberate concession to the deployed base, and each narrows what the protocol can change later. The fixed version bytes mean the record layer cannot signal a new version in the place a middlebox looks. The dummy ChangeCipherSpec means the record sequence retains a vestigial message that has no cryptographic function. The specification pays those costs because the alternative, a protocol that devices drop, is worse.

HTTP/3’s deployment shows the complementary case. Because it runs over QUIC and UDP, networks that block UDP or that rate-limit unfamiliar traffic make it unusable, and the fallback to HTTP/2 over TCP is what keeps those clients working. The protocol’s extensibility is not the constraint there; the path’s willingness to carry the transport is. That is ossification at the transport layer rather than the message layer, and the remedy is a fallback rather than a change to the protocol.

The lesson for design is that extensibility is not a property that can be established once. It has to be maintained against a network that is constantly adding devices, and the maintenance is a continuing cost in encryption, in visible-field discipline, and in the willingness to break devices that refuse to skip. The mechanisms that make the maintenance cheaper are the ones that make skipping the default and make not-skipping fail during development, which is what reserved values are for.