Normative and Informative Text in a Specification

A specification is two documents interleaved. One states obligations; the other explains them. The interleaving is not decorative, and reading the explanatory passages as though they were obligations produces implementations that fail conformance tests for reasons no clause actually required. Reading the obligations as though they were explanations produces the opposite defect, silently: a product that works against one peer and not another.

The vocabulary that separates the two is itself standardized, and its history explains why the separation is as awkward as it is.

The keyword list and its capitalization rule

RFC 2119 defines a small set of words that, when they appear in a specification, carry requirement semantics. The uppercase forms are the ones that count: MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL. Lowercase uses of the same words carry no special meaning and are ordinary English.

The definitions are worth stating precisely because the middle of the list is where implementers misread.

MUST, REQUIRED and SHALL mean that the definition is an absolute requirement of the specification. An implementation that does not do it conforms to nothing. MUST NOT and SHALL NOT are the corresponding absolute prohibitions.

SHOULD and RECOMMENDED mean that there may exist valid reasons in particular circumstances to ignore a requirement, but the full implications must be understood and carefully weighed before choosing a different course. This is not a weaker MUST. It is a burden of justification, and it is the only category in which an implementation can be conformant while behaving contrary to a sentence describing expected behaviour.

SHOULD NOT is the same structure applied to forbidden behaviour. MAY and OPTIONAL mean that an item is genuinely optional: an implementation which includes the item must be prepared to interoperate with one that does not, and vice versa. That clause turns MAY into a bilateral obligation, which is why optional features that both sides skip are the most reliable form of interoperability.

Capitalization created a loophole that took years to close. RFC 2119’s original text gave the words their force when they appeared in capitals, and a specification that didn’t announce the convention left readers unsure whether ‘must’ in sentence two carried any weight at all. RFC 8174 later clarified that the keywords have their special meaning only where the document explicitly says it uses them, which makes the boilerplate paragraph an actual enabling clause rather than a formality.

Why the split exists at all

The separation of requirement from explanation has a testability motive. A conformance test must produce a verdict, and only a normative clause can be the subject of one. A sentence that says a peer ‘will normally retry after a delay’ is not testable: an implementation that never retries has not violated anything because the sentence described a tendency, not a duty.

The split also protects the committee from itself. Background material can be revised freely in later editions because it carries no obligations; the normative clauses are what require a ballot to change. In a document that grows over a decade, most of the prose volume rarely changes status, and most of the change is confined to a small set of numbered requirements.

The cost is duplication. The explanatory passage and the requirement it explains drift as the document is edited, and where they conflict the requirement wins even when the commentary is more recent and better informed. Implementers who read only the narrative sections inherit the drift.

Reading a clause for what it binds

A normative sentence has a subject, an object, and often an unstated precondition. The precondition is the part that gets lost.

Take a clause in the form “an implementation MUST reject a message whose length field exceeds the received octets”. The obligation is conditional: it binds only when the length field exceeds the received octets. A message whose length field is exactly equal to the received octets is not covered, and neither is one whose length field is smaller. What the receiver does in those cases is unspecified, which means any behaviour is conformant, which means peers cannot rely on it.

Specifications that anticipate this add a default rule: anything not covered by a requirement is unconstrained. Others state a conservative principle instead, requiring that unspecified behaviour at least not crash the process. The difference matters at the boundary, where most real defects live.

Numeric quantities sharpen the same point. A requirement that a header be “at least 12 octets” permits 12, 13, and 40. A requirement that it be “12 octets” forbids anything else. A requirement that an identifier be “unique” offers no window at all and usually means the author did not decide. When a specification uses a vague quantifier inside a MUST, the implementer has to choose, and the choice becomes de facto law — the dynamic described where de jure and de facto standards diverge.

Extension points and the MUST-ignore rule

One recurring pattern shows the split doing real work. A base protocol defines a field that a future version will extend, and the current text requires receivers to ignore values they do not recognize. The requirement is normative and testable: a receiver given an unknown value MUST behave as if the value were absent, not reject the message.

That single rule is what allows a protocol to add optional behaviour without version negotiation. It also constrains what the base version may send: if a base implementation must ignore an unknown value, then the sender cannot rely on the receiver doing anything in particular with it, so any meaning attached to the unknown value has to be optional by construction. The rule converts forward compatibility into a one-directional obligation.

The grey zone: SHALL in annexes and diagrams

Documents routinely carry normative material outside their numbered clauses. A figure may contain a state transition that no prose describes; an annex may be labelled normative at the front; a profile, which is a separate document, may make one of these clauses mandatory where the base left it optional, as covered in the discussion of profiles and conformance classes. Each of these is easy to miss, and each binds.

The front matter is the authoritative statement of what binds: a scope clause and a conformance clause that names which parts of the document are normative. Where a document does not carry such a clause, the safe reading is that everything binds except explicitly marked background. Where it does, the marker governs regardless of where the text sits.

A worked hazard: a table that maps status codes to meanings, introduced in a background section and captioned as illustrative, and referenced by later behaviour. The table is informative. A client that keys off it is relying on guidance, not requirement. When a later revision changes the table, the client’s assumption is invalid even though no conforming implementation broke.

Requirement levels in documents that do not use the keyword list

Not every standards body uses RFC 2119 vocabulary, and a reader moving between bodies has to translate. ISO and IEC drafting rules use a verb system instead: “shall” for requirements, “should” for recommendations, “may” for permission, and “can” for possibility or capability. The mapping to the IETF list is close enough that the same reading applies, with one trap. In ISO and IEC text, “may” expresses permission granted by the document, which is not the same as the IETF’s MAY when the latter describes an optional feature that peers must interoperate with. A permission (“the implementation may cache the response”) and an optional capability (“the implementation may include the field”) look identical in English and impose entirely different obligations.

The other trap is the passive construction. “The value shall be ignored” and “the receiver shall ignore the value” are the same obligation stated from opposite ends, but only the second names a party that can be tested. Drafting guides recommend the active form for exactly this reason. A specification that states obligations without naming the actor leaves the question of who must implement the check open, and in a protocol with two sides, both may assume the other.

Modal verbs inside tables

Tables are compact and therefore tempting, and a requirement written into a table cell loses its modal verb rather than gaining precision. A row that reads “Reserved — implementations ignore” does not say whether the ignoring is mandatory or a description of current behaviour. Where a table carries requirements, the surrounding text needs an explicit statement of what its rows bind; that statement is what a conformance suite will later have to encode.

Checking conformance against the text

An implementation review that wants to be meaningful enumerates normative clauses and asks, for each, whether the implementation satisfies it under the stated preconditions. The exercise tends to produce three buckets: clauses satisfied, clauses with unclear preconditions, and clauses whose object is a vague predicate. The middle and final buckets are where the specification, not the implementation, is the defect.

That is also the practical argument for precise requirement language. The point of the normative split is not bureaucratic tidiness: it is that a claim an implementer cannot test is a claim that will be resolved by whichever implementation ships first, and the specification’s authority erodes with each such resolution.