Proactive Content Negotiation and the Quality Value Grammar

A server holding several representations of one resource has to choose one per request. HTTP offers two mechanisms. Proactive negotiation has the client state its preferences in the request and the server pick before sending anything. Reactive negotiation has the server send a list and let the client choose, which costs an extra round trip and is rarely used. The proactive form is the one in general use, and its grammar has properties that surprise people who read it casually.

The Accept grammar

Accept lists media ranges with optional parameters, separated by commas. A range is either a concrete type and subtype (text/html) or a wildcard form: text/* matches any text subtype, and */* matches anything. Parameters after a semicolon narrow the range, so text/html;level=1 matches only representations declaring that parameter.

The subtlety is that the media range’s own parameters and the q-value share the same syntactic position. RFC 9110 resolves this by giving q a fixed meaning: it is the weight parameter, and it always separates the range’s media type parameters from extension parameters that follow it. In text/html;level=1;q=0.7;foo=bar, everything before q constrains the match and everything after q is an accept extension with no effect on selection. A parser that treats q as just another parameter will mis-rank ranges that carry both.

Quality values range from 0 to 1 with up to three decimal places: 0, 0.5, 1, 0.001. A q-value of 0 means not acceptable, which is a positive exclusion rather than an absence of preference. Accept: */*;q=0.1, text/html;q=0 says any type is tolerable at low priority except HTML, which must not be sent. Servers that treat q=0 as merely a low score will send the one type the client explicitly refused.

Default weight is 1 when no q appears, and a range with no q outranks any range with a q below 1. The specification also gives a precedence rule for matching: the most specific range that matches a representation determines its quality. A request with Accept: text/*;q=0.3, text/html;q=0.7, */*;q=0.5 assigns text/html a quality of 0.7, text/plain a quality of 0.3, and image/png a quality of 0.5. Specificity is measured by the number of non-wildcard components and parameters, not by position in the list.

Language matching is not string equality

Accept-Language uses the same q-value syntax with language ranges instead of media ranges, and the matching algorithm is defined by RFC 4647 rather than by HTTP. Basic filtering treats each range as a prefix: en matches en, en-US, and en-GB, because a range matches a tag that begins with it and continues with a hyphen or ends there. en-US does not match en; the range is more specific than the tag.

The consequence for servers is that a lookup table keyed on exact tags is wrong. A client asking for en-GB matches nothing in a server that holds only en, because basic filtering extends a range downwards, not a tag upwards; a real implementation may still choose to fall back to the less specific tag. The reverse case, a client asking for en and a server holding en-US and en-GB, requires a choice among equally matching tags, and the specification does not rank them; any deterministic tie-break is conformant.

The wildcard * in Accept-Language matches any tag, and its q-value sets the floor for everything not otherwise listed. A request with no Accept-Language at all conveys no preference, which is different from a request with Accept-Language: *; both permit any language, but the second states that any language is acceptable while the first states nothing. The difference is visible to caches, since Vary on a field that is absent produces a different key than Vary on a field that is present with a wildcard value.

Encoding negotiation is a three-way choice

Accept-Encoding is the field with the most operational weight, because it decides whether the response is compressed and therefore how many bytes cross the network.

The grammar permits codings, *, and q-values. Two behaviours are worth stating precisely. First, identity, meaning no transformation, is always acceptable unless explicitly refused with identity;q=0 or excluded by *;q=0 without a more specific identity entry. A server that cannot produce an acceptable coding may still send identity, which is the default fallback; a request that refuses identity as well as every coding the server can produce leaves 406 as the only honest answer. Second, the absence of the field means any coding is acceptable, whereas an empty field value means only identity is acceptable. That inversion trips up implementations that initialise the header to an empty string.

A server choosing among codings it supports should pick the highest-quality one it can produce, breaking ties by its own preference. Compression level is not negotiated; the client cannot ask for a specific ratio, and the server’s choice of level is invisible to the protocol.

The interaction with caching is direct. A response encoded with a coding the cache does not understand may still be stored but must not be served to a client that did not accept it. This is why caches commonly store the identity representation and compress on the fly, trading CPU at the edge for storage of a single variant.

Vary is what makes negotiation correct

A negotiated response is not a single resource representation; it is one of a family. A shared cache that keys only on the request URI will store the German HTML page and serve it to an English client. Vary fixes this by naming the request fields that participated in selection. The cache then stores the response under a secondary key derived from those fields’ values in the original request, and a later request matches only if its values are the same.

Vary: Accept-Encoding is the common case and is nearly mandatory for any compressible response. Vary: Accept-Language and Vary: Accept follow. Vary: * means the response could not be reused for any other request, which effectively disables caching for that resource and is the honest declaration for a response that depends on something the server cannot name.

The cost of Vary is multiplicative. Each listed field multiplies the potential number of stored variants by the number of distinct values that field can take, and a cache with a limited entry budget facing Vary: Accept, Accept-Language, Accept-Encoding will evict variants faster than it fills them. Varying on User-Agent for a trivial reason produces exactly that: the variant count explodes, hit rate collapses, and the origin sees nearly as much traffic as it would with caching disabled. RFC 9111 allows a cache to normalise field values before using them as a key, sorting a list for instance, and implementations differ in how aggressively they do this, which means a normalisation that changes which request matches which stored response is a correctness bug waiting to be found.

Vary also interacts with conditional requests and entity validators. An entity tag identifies a specific representation, not the resource, so a client that revalidates after changing its Accept field may legitimately receive a 200 with a different representation rather than a 304. A server that compares the tag alone and returns 304 has told the client its cached copy is current when it is the wrong variant.

What the server does with all this

The selection procedure is a ranking, not a first match. For each available representation, compute its quality against each negotiating field, then combine. The specification does not define the combination, because the relative importance of media type, language, and encoding is application-specific. A server that multiplies the three q-values gets a defensible ordering; one that filters on media type first and then sorts by language gets a different and equally defensible one.

Two failure modes recur. The first is a server that ignores the request entirely and always sends its default representation, which is conformant only if the client accepts that representation. The second is a server that returns 406 whenever it cannot satisfy the exact request, when the specification’s expectation is that a server with any acceptable fallback should send it. 406 is reserved for the case where nothing the server can produce is acceptable, and that case is rare because */* and implicit identity acceptance cover most requests.

When negotiation genuinely cannot be done in one pass; because the choice depends on a computation the server cannot do until it has the body, or because the variant set is too large to enumerate; the alternative is to send 300 Multiple Choices or to redirect to a variant-specific URI. The second avoids the cache-key problem entirely: distinct URIs per language, chosen by the server from Accept-Language, with the negotiation result encoded in the URL so that the response itself no longer needs Vary.