Critical Bits and Unknown-Field Handling

A protocol with a length field can carry a field it does not understand. That says nothing about whether it should accept one. The two decisions are independent, and conflating them is where extensibility designs fail: a parser that skips everything unknown is compatible with the future but has no way to refuse a change it cannot safely process, while a parser that rejects everything unknown is safe in principle and permanently frozen in practice.

The standard resolution is a per-field bit that the sender sets to declare a field essential. The receiver’s behaviour then depends on a two-dimensional table: whether it understands the field, and whether the sender marked it essential. Four cells, and every protocol that has solved this problem fills them differently.

The four cells and the one that matters

Take the easy cells first. A field the receiver understands is processed according to its semantics whether or not it was marked essential; the bit is redundant for known types, and an implementation that behaves differently based on the bit for a known type is buggy.

A field the receiver does not understand and the sender marked non-essential is skipped. This is the cell that makes extension possible, and its correctness depends entirely on the skip being sound: the length must be trusted only after being validated against the container, and skipping must not have side effects.

A field the receiver does not understand and the sender marked essential is the interesting cell. The receiver cannot process it, so it cannot continue safely. The only correct response is to fail the whole message, not to fail the field, because a partial parse of a message that declared a dependency has undefined meaning. Failing the field and continuing is worse than failing the message: the sender believed something was established, and the receiver silently operates without it.

The fourth cell is a field the receiver understands but that is marked essential and whose value it considers unacceptable, which is a plain semantic error and not an extensibility concern.

The design lever is the default. A one-bit flag means the default is one of the two values, and the choice of default determines which mistake is easy to make. If fields default to essential, an implementer adding a field must remember to clear the bit for anything optional, and forgetting it breaks every peer that has not been updated. If fields default to non-essential, an implementer must remember to set the bit for anything that changes interpretation, and forgetting it produces silent misbehaviour in old peers.

X.509’s certificate extensions carry an explicit critical flag, and the default is non-critical: an extension that a validator does not recognise is ignored unless the issuer set the flag. A CA that adds an extension changing how the certificate may be used must therefore set the flag deliberately, and a validator that does not understand a flagged extension must reject the certificate. The name is a trap for readers: “critical” in X.509 means “must be understood”, not “important”. An extension that is important but ignorable is marked non-critical, and the default makes the ignorable case the one an issuer gets without thinking.

DNS takes a third position: there is no critical bit at all. An unknown resource record type in a response is simply not used by a resolver that does not recognise it, and the type code space is the only signal. The design consequence is that a DNS extension cannot demand to be understood, so an extension that changes how a response must be interpreted cannot be introduced at the record level at all; it has to be introduced as a change to the query or as a new class of response that old resolvers will fail to match. EDNS handles the negotiation part by putting an options list in an additional record and letting the sender state its buffer size, which is the closest DNS comes to a capability advertisement.

When a bit is not enough

A single bit per field is the minimum, and it is insufficient whenever the meaning of one field depends on another. Suppose an extension alters the interpretation of a field that already exists. Marking the extension essential is enough for a receiver that understands the extension. But a receiver that understands the base field and not the extension will parse the base field under the old interpretation, and if the extension is marked essential it will reject the message, which is correct. If the extension is marked non-essential, the receiver proceeds with the old interpretation, and now the sender and receiver disagree about what the base field means. The non-essential marking was a lie: the extension was essential to the base field even though it was optional for the message.

Protocols handle this by making some dependencies structural. TLS 1.3 requires that a server not select a cipher suite the client did not offer, and that any extension the server sends in ServerHello must have appeared in ClientHello. Those rules are not expressed as critical bits; they are expressed as a prohibition on the server inventing extensions unilaterally. The client’s offer is the negotiation, and the server’s response is constrained to a subset of it. The cost is one round trip of state on both sides; the benefit is that no unilateral extension can ever appear in a message the peer must interpret.

HTTP/2 took a third approach for its settings: an unknown setting in the SETTINGS frame is simply ignored, and the frame’s length field lets the receiver skip it. There is no critical bit, because a setting that must be understood would break the “ignore unknown” rule that makes the frame extensible, and the protocol instead provides a way to advertise what is supported. When the advertisement is absent, the default applies. The encoding that makes this skip possible is the type-length-value layout applied to a fixed frame header.

The general lesson is that the critical bit handles a field whose presence can be ignored. It does not handle a field whose absence changes the meaning of something else. For that, the protocol needs a statement about which combinations are legal, and legality is not expressible in one bit.

Rejection has to be cheap and specific

Failing a whole message because of one unrecognised essential field is correct but blunt. It also has a bad interaction with negotiation: if a client sends an essential extension and the server rejects the message, the client has no way to know whether the server rejected the extension or something else. The client may retry without the extension, or it may give up, and both are guesses.

The remedy is a diagnostic channel that carries the reason for the failure, and this is where the design becomes concrete. TLS has an alert protocol: a two-byte message, one byte of level and one byte of description. An implementation that rejects an essential extension can send unsupported_extension rather than closing the connection silently. The peer learns which class of problem occurred, though not which extension, because the alert vocabulary is coarse. A finer channel would be more useful and would also be more state to maintain across versions.

ICMP performs the same role for IP, with the same coarseness: destination unreachable, parameter problem, and a code field that narrows the cause. The original ICMP parameter problem message carries a pointer to the byte offset of the offending field, which is a genuinely useful diagnostic and also a small information leak about the receiver’s parser layout.

The cost of a diagnostic channel is that it is itself a protocol with its own extension problem. TLS alerts have a registry, and new alert codes are rare because every implementation must decide what to do on receipt. The practical rule is to define a small set of failure classes at design time and accept that the diagnostics will be less specific than the implementation knows.

What “ignore unknown” costs the sender

Every extensible protocol eventually discovers that the ability to ignore is not symmetric with the ability to rely. A sender that includes an optional field cannot know whether it was honoured. The receiver skips it and sends back a response that looks identical to the response it would have sent without the field. The sender therefore cannot use the field for anything that requires confirmation.

This is the reason protocols grow acknowledgement mechanisms for their extensions. A client that wants to use an optional feature can include it and then look for evidence in the response: an echo of the field, a flag, a different behaviour. If the evidence is absent, the feature is unavailable. That pattern costs a round trip in the cases where it is used, which is why it is usually reserved for features where the fallback is materially different rather than for cosmetic additions.

A cheaper pattern is to advertise capabilities once and cache the result. HTTP’s Upgrade mechanism does this with a request header and a response status, at the cost of one request that is dedicated to the negotiation. TLS session resumption caches the negotiated parameters so that the second connection does not repeat the exchange.

Both patterns share a property: the sender learns the receiver’s capability set, not just whether one message was accepted. That distinction is what separates an extension that can be relied upon from one that can only be hoped for. The design of that learning step, and the number of round trips it costs, is the subject of negotiation round trips and their cost.

Silent acceptance is the failure mode to test for

The bug that survives review is the parser that accepts an unknown essential field because the check was placed after the parse, or because the unknown-field path was never exercised. Testing for it requires a message with a field the implementation does not know, marked essential, and an assertion that the connection fails. Most test suites do not include such a message, because generating one requires knowing the registry well enough to pick an unallocated number, and the number must be unallocated at the time the test is written. The reserved-value ranges exist partly to solve that problem: they provide numbers that are guaranteed unallocated and that a conforming receiver must ignore.

The complement is equally necessary: a message with an unknown non-essential field must be accepted and must produce exactly the behaviour it would have produced without the field. Testing both directions catches the two common errors, which are rejecting too much and rejecting too little.

There is a third case that is harder to test and more damaging in practice: a field that is understood but whose handling is inconsistent between the two directions of a connection. A client that processes a field and a server that ignores the same field will disagree about the state of the connection, and the disagreement may not surface until much later. The protocol cannot prevent this; only implementation discipline and shared test vectors can. That is why specifications that care about it publish conformance suites rather than prose alone, and why the sections on unknown-field handling tend to be the most prescriptive parts of an otherwise permissive document.