Heading Anchors and the Silent Failure of Fragment Links

Documentation URLs are cited in tickets, chat threads, source comments, and other documentation. Most of those citations point at a page, and a page that moves is a link a redirect can repair. A growing share point at a section within a page, and that kind of link has no repair mechanism at all: when the heading is reworded, the reader lands at the top of a page that still exists and has to search it by eye. Nothing in the server log records the failure.

The reason is structural rather than accidental. RFC 3986 splits a URI into scheme, authority, path, query, and fragment, and the fragment is defined as identifying a secondary resource within the primary one. HTTP never transmits it. A request for https://example.org/guide/#retry-policy arrives at the server as a request for /guide/, and the browser applies #retry-policy locally after the response is rendered. From the server’s point of view, a million readers arriving at a dead fragment and a million arriving at the top of the page are indistinguishable.

The fragment is matched against ids in the rendered document

After parsing, the browser looks for an element whose id attribute equals the fragment, comparing after percent-decoding the fragment and decoding it as UTF-8. The comparison is case-sensitive, so #Retry-Policy does not match id="retry-policy". If no element matches, HTML defines a fallback: a fragment that matches top case-insensitively scrolls to the document start, and the empty fragment does the same. Every other miss leaves the reader at the top with no indication that anything went wrong.

Two encoding details produce failures that look like typos in the source. A # character inside a URL always begins the fragment, so a heading whose slug would contain # must percent-encode it as %23 in the link, and a link that writes it literally is truncated at that point. Non-ASCII characters may appear directly or percent-encoded, but the decoded bytes must match the id exactly, which brings Unicode normalisation into play: a heading stored with a decomposed accented character and a link written with a precomposed one are different byte sequences and different ids, even though they render identically.

The document itself has one hard constraint: ids must be unique. The HTML parser does not reject duplicates, and the scrolling algorithm takes the first matching element in tree order, so a duplicated id silently redirects readers to the wrong section. Duplicates usually enter through generated pages, where an id from a template collides with one derived from a heading.

Slugs are a lossy function of heading text

Static site generators derive the id from the heading text, because asking authors to invent one per heading is friction. The derivation is a slug: lowercase the text, strip punctuation, replace runs of whitespace with a hyphen. GitHub’s Markdown renderer is a widely copied implementation, and its convention is visible in every rendered README: ## Retry policy and backoff becomes #retry-policy-and-backoff.

Slug functions differ between tools in ways that matter when a site changes generators. Some strip all non-ASCII, some preserve it; some collapse consecutive hyphens, some do not; some keep underscores, some treat them as word separators; some strip a leading digit sequence. A migration between generators can therefore rewrite every anchor on the site at once, and the rewrite is invisible in the rendered output because the headings look the same.

The loss is the point worth stating. The slug is derived from the text, so editing the text changes the identifier. Words added for clarity break links that were made before the clarification. This is inherent to deriving an identifier from content: the identifier inherits every change to the content, and headings are edited far more often than URLs.

A compact implementation shows where the choices sit:

import re, unicodedata

def slug(text, seen):
    text = unicodedata.normalize('NFKD', text).encode('ascii', 'ignore').decode()
    text = re.sub(r'[^\w\s-]', '', text.lower())
    text = re.sub(r'[\s_]+', '-', text).strip('-')
    n = seen.get(text, 0)             # duplicate headings get -1, -2, ...
    seen[text] = n + 1
    return text if n == 0 else f'{text}-{n}'

The seen dictionary is not an optimisation. It is the mechanism that keeps ids unique within a page, and it is also a source of instability discussed below.

Duplicate headings renumber the anchors after them

When two headings produce the same slug, generators append a counter: the second occurrence becomes retry-policy-1, the third retry-policy-2. The suffix is positional. Inserting a new heading with the same text before an existing one renumbers every later duplicate, so links that pointed at retry-policy-2 now land on what used to be retry-policy-1.

The same fragility appears in the class of pages where duplicates are normal: reference pages with one section per overload, changelogs with one Fixed heading per release, and API pages with a ## Parameters heading for every function. For those pages the anchor is effectively a pointer into an ordered list, and reordering the list rewrites the anchors.

Two arrangements avoid it. The first is to disambiguate the heading text itself, so that no two headings on a page share a slug; a page with Parameters twice is better served by Parameters of open and Parameters of close, which also reads better. The second is to stop deriving the id from the text and let the author name it.

Explicit labels move the identifier out of the prose

Sphinx and several other documentation toolchains separate the two concepts. An explicit target, written as a label above a heading, is the identifier; the heading supplies the display text. References point at the label. Renaming the heading or moving the section to a different page leaves the label untouched, and the link still resolves.

.. _retry-policy:

Retry policy and backoff
========================

Clients retry when the response carries a ``Retry-After`` header...

The :ref: role then resolves retry-policy at build time and reports an undefined label as a warning, which becomes a build failure wherever warnings are treated as errors. That converts a silent reader-facing failure into a build-time one. Markdown-based generators offer an equivalent mechanism under a different name: an explicit id attached to the heading, written as a trailing {#retry-policy} in several implementations. The mechanisms differ; the shape is the same. An identifier that is not derived from display text is the only kind that survives editing.

Indirection has a cost, and it is the same cost every indirection has: the label namespace is global and must be curated. Two authors writing retry-policy for different sections produce a collision that the build detects, and the fix is a rename that will itself break any external link that used the old anchor. Labels need a naming convention with a page prefix, decided once.

Checking fragments means parsing the rendered pages

A conventional link checker requests each URL and reports non-2xx responses. That catches moved pages and nothing else. Catching dead fragments requires a second pass over the built site: collect the set of ids on each page, collect the fragment of every internal link, decode both, and compare.

The second pass has three requirements that are easy to miss. It must run on the output of the generator, because the ids are produced by the renderer rather than written in the source. It must handle the difference between a fragment that names an id and one that names an element by name, which the HTML scrolling algorithm also accepts. And it must be able to distinguish an intentional external link from an internal one, since only internal fragments can be verified.

The check is deterministic and cheap, which makes it a good candidate for a build gate rather than a periodic report. The failure it prevents is otherwise undetectable: a page that renders correctly, serves a 200, and sends the reader to the wrong paragraph. Deterministic documentation checks covers the wider family of assertions that can be made about a built site; fragment resolution is the one that most often goes missing because the naive checker appears to be doing the job.

Anchors are an interface with a retention obligation

The uncomfortable conclusion is that a published anchor is a public identifier, and treating it as a rendering detail is what causes the breakage. Two policies follow, and a site should pick one explicitly.

The first is that anchors are permanent: once published, an id is never removed, only aliased. Renaming a heading leaves an empty anchor element carrying the old id, and the number of such elements grows without bound. The second is that anchors are best-effort, in which case section-level citations should not be encouraged and the site should not pretend they are stable.

Between the extremes sits the arrangement most large documentation sites converge on: explicit labels for sections that are cited from outside, generated slugs for the rest, and a redirect table for pages that move. The redirect table cannot help with fragments, because no request is made, but the browser does carry the original fragment across an HTTP redirect when the Location value has none of its own, so a page-level redirect at least preserves the section pointer. That inheritance is the reason moving a whole page is cheaper than renaming one heading on it, and it is why restructuring a documentation version needs a redirect entry per old path even when the content is unchanged.

None of this makes anchors stable by itself. It makes the instability visible at the moment it is introduced, which is the most a link can ask for.