HTML Parsing Contexts and Why One Escape Function Is Never Enough

A web page is assembled from many sources, and most of them are not trusted to produce markup. The templating layer therefore has to make untrusted text inert. The usual mistake is to treat that as one problem with one solution: a function called escapeHtml that replaces five characters, applied wherever untrusted text appears. That function is correct for exactly one of the positions where text can appear, and silently wrong in the rest.

The reason is that HTML has no single lexer. The tokenizer in the HTML Living Standard is a state machine with dozens of states, and the same input character has different meaning depending on which state is current. Escaping is the act of converting a character into something the current state will treat as data. If the state is wrong, the conversion is aimed at the wrong target.

The tokenizer as a state machine

The tokenizer begins in the data state. In that state, < is the only character that changes anything: it moves to the tag-open state. Everything else is character data emitted to the tree builder. An escape function for this state must therefore neutralise <, and the standard encoding is <. The ampersand is also encoded, as &, because an unencoded & followed by a name and a semicolon would be decoded as a character reference — the tokenizer’s character-reference state runs inside the data state, so leaving & alone would let input produce a < after decoding, defeating the < encoding. > is typically encoded as well, though it is not strictly necessary in the data state; it matters in unquoted attribute values and in comments.

Once < has been consumed, the tokenizer is in a tag state. Attribute values have their own sub-states: before-attribute-value, attribute-value-double-quoted, attribute-value-single-quoted, and attribute-value-unquoted. Inside a double-quoted attribute value, the quote character ends the value; < and & are still active. Inside an unquoted attribute value, space, >, and several other characters terminate the value. A string placed into a quoted attribute needs " (or '), &, and < encoded; a string placed into an unquoted attribute needs every character that can terminate the value encoded, which is a longer and less obvious set.

Script and style elements are stranger still. Inside <script>, the tokenizer is in the script data state, and character references are not decoded at all: < inside a script element arrives at the JavaScript parser as the literal five characters <. HTML-escaping a value inside a script block therefore does nothing useful for the JavaScript grammar, and the relevant danger is </script, which returns the tokenizer to tag-open regardless of JavaScript string quoting. The correct treatment for data destined for a script block is not HTML encoding but JavaScript string encoding, followed by ensuring that no </script sequence survives — usually by encoding < as \u003C in the JavaScript literal.

Why context, not character class, is the unit

An escaping routine that encodes <, >, &, ", and ' is safe in the data state and in a quoted attribute value. It is unsafe in an unquoted attribute value containing a space, and it is unsafe in a script block. The failure is not that the function encodes too little; it is that the function encodes for the wrong grammar.

Consider a URL in an href. Encoding quotes protects the attribute, so the value cannot break out of the attribute — but the value is then parsed as a URL, and a javascript: scheme is a valid URL. Attribute-level escaping does not address the URL grammar. A URL sink needs scheme validation against an allowlist, and the validation has to run on the value after any percent-decoding the browser will perform, because java%0Ascript: and similar constructions are attempts to have the scheme comparison disagree with the browser’s parser.

The same layering recurs in CSS, in srcset, in meta refresh targets, and in SVG. Each is a separate grammar nested inside the HTML attribute, and each has its own notion of an executable value. The practical consequence is that an escaping function is only meaningful when paired with the exact sink it will feed, which is why output encoding in modern frameworks is done at the point of interpolation by a context-aware auto-escaping layer rather than by a helper the developer remembers to call.

Auto-escaping and its blind spots

Template engines that auto-escape do so by parsing their own template source, determining the static context surrounding each interpolation, and applying the matching encoder. A value interpolated between tags gets HTML encoding; a value inside a quoted attribute gets attribute encoding; a value inside a <script> block gets JavaScript encoding. This is a static analysis of the template text, and its correctness depends on the template being parseable as HTML at the point where the interpolation sits.

Blind spots follow directly from that dependency. An interpolation inside an attribute that is constructed dynamically, where the attribute name itself comes from data, defeats the analysis. An interpolation passed through a filter that marks the result as pre-sanitised removes the escaping entirely; the filter is a promise that the developer made and the engine cannot check. A template that composes another template’s output as a string, then inserts that string, is analysed as one unit only if the engine sees the composition.

The |safe style of marker deserves separate mention because it is load-bearing in real code. It exists to say “this value is already correct markup”. When the value is produced by a sanitiser, the marker is honest, and the sanitiser must then be the thing that is correct — a sanitiser is a parser, not a regex, and the HTML sanitisation problem has a specification. When the value is produced by string concatenation of user input, the marker is a claim the engine cannot verify and the browser will not check.

Sanitisation as a different tool

Escaping and sanitisation solve different problems and are not interchangeable. Escaping is for a value that must remain a value: a comment body, a display name, a search term. Sanitisation is for a value that is permitted to be markup: a rich-text comment with bold and links, an email body rendered in a webmail client.

A sanitiser parses the input as HTML, walks the resulting tree, and removes elements and attributes not on an allowlist. Because it parses, it is immune to the encoding tricks that defeat string matching: <scr<script>ipt> is parsed as a text node containing <scr followed by an ipt> element, not as a script element, and a tree-based sanitiser handles that correctly while a regex-based one does not. Because it must also handle the browser’s error recovery, it must implement the same tree construction the browser does, including the rules that move elements out of table contexts and close implied tags.

The residual difficulty is that a sanitiser runs on the server and the browser parses the same bytes with the same specification, but the two implementations can differ at the edges. Mutation XSS is the class of bugs where a sanitiser’s output, when re-parsed by a browser, has a different tree than the one the sanitiser approved. The general shape: the sanitiser serialises a tree, and serialisation followed by re-parsing is not the identity for all inputs. Attribute values containing entity-like text, or namespace confusion between HTML and SVG, are the usual triggers.

The layer above

Content Security Policy is the control that catches the cases the encoding layer missed. A policy that forbids inline script and restricts script-src to a known set of origins turns a successful injection into a blocked request, provided the policy does not itself admit a bypass. Nonce-based policies and the script gadgets that defeat them covers where that proviso fails.

Encoding remains the primary control, because it prevents the injection rather than detecting it. The discipline is to name the sink before choosing the encoder: text node, attribute value, URL, script body, style body, or markup that is allowed to be markup. Six sinks, at least four distinct encoders, and one sanitiser. A single escapeHtml covers one of them.

Encoding that never runs is the other half of the problem: values that a script reads from the DOM and reinserts bypass the encoder entirely, which is the flow traced in DOM XSS sources and sinks.