Verifying What a Screen Reader Actually Says

A screen reader is a program that watches a platform accessibility API for events and turns them into speech. It is not a program that reads the DOM. The distinction explains both why announcement tests are valuable, because they test the actual output, and why they are fragile: the speech is a rendering, and the renderer has its own conventions, its own settings, and its own release schedule.

An assertion like “the reader says ‘Filters, button, collapsed’” is an assertion about a specific reader at a specific version with a specific verbosity configuration. An assertion like “the tree exposes a button named Filters with expanded false, and a state-change event fires when it is activated” is an assertion about the platform, and it survives reader upgrades.

From tree to speech

The path has four stages, and a defect can enter at any of them. The browser projects the DOM into an accessibility tree and exposes it through the platform API. The screen reader subscribes to events on that API: focus changes, name changes, state changes, subtree creation, selection changes, live region updates. It maintains its own model of the document, a virtual buffer in the case of browse mode, which is a snapshot plus a set of rules for how to move through it. It renders the model into an utterance, applying conventions: roles are announced after names, states are announced when they differ from a default, punctuation is filtered by level, and repeated content may be suppressed. The synthesizer turns the utterance into audio.

The virtual buffer is the reason browse mode and focus mode behave differently. In browse mode, the reader builds a linear representation of the tree and navigates it with single-letter commands and arrow keys; the browser’s own focus does not move, and the reader’s cursor is a position in its model. In focus mode, keystrokes pass through to the page and the browser’s focus moves. A widget that requires arrow keys to operate must put the reader into focus mode, either by being a role the reader recognises as interactive or by an explicit request from the page. A page that fails to do this is operable by mouse and by a reader in focus mode and not operable by a reader in browse mode, and the failure is invisible to every check that reads the DOM.

Capturing output

Screen readers expose their speech in a viewer: a window that shows the text as it is synthesized, with the order and the pauses. This is the practical capture mechanism. It is a human-readable log rather than a machine-readable stream, which means automated capture requires reading the viewer’s content through the platform API or parsing an exported log, and both are version-dependent.

There are two more capture points that are more stable. The first is the browser-level view of the tree and its live region attributes, which Chrome DevTools exposes in its accessibility panel, so that a change can be observed as a tree mutation rather than as speech. This tests the browser, which is the layer the page controls. The second is the synthesized audio, which on each platform goes through a system speech API that a test harness can substitute or intercept. Substituting the synthesizer gives a clean text stream but removes the reader from the loop, so it tests the utterance construction only if the reader is the thing being intercepted.

The choice of capture point determines what the test can claim. Intercepting the synthesizer tests the reader’s utterance construction. Reading the viewer tests the same thing with a human in the loop. Observing the tree mutation tests the browser’s projection, which is upstream of the reader and is the layer a page can actually control.

What is stable enough to assert

The properties that hold across readers and versions are the ones defined by the specifications: the role, the computed name, the states, the relations, and the events. Everything about wording is a convention.

Wording conventions that vary: whether a role name is spoken at all in browse mode, whether the level of a heading is announced or only the fact of it, whether “collapsed” or “not expanded” is used for a disclosure, whether a list announces its item count, whether punctuation is spoken, and whether a group’s name is announced before or after its contents. None of these are specified, all of them change between versions, and several are user-configurable.

What is specified is the mapping. A <button aria-expanded="false"> maps to a button role with an expanded state of false. A <h2> maps to a heading role with level 2. A <input type="checkbox" checked> maps to a checkbox with a checked state. A live region with aria-live="polite" maps to a polite live region, and a text insertion inside it produces an announcement event. These are the assertions that a test suite can hold.

// The stable contract: role, name, and state, read from the projection.
const findNode = (n, role, name) =>
  (n.role === role && n.name === name) ? n
  : (n.children || []).map(c => findNode(c, role, name)).find(Boolean) || null;

const snapshot = await page.accessibility.snapshot();
const node = findNode(snapshot, 'button', 'Filters');
assert.equal(node.role, 'button');
assert.equal(node.name, 'Filters');
assert.equal(node.expanded, false);
// Wording of the announcement is the reader's business, not the page's.

Live regions and the timing problem

Live regions are where announcement testing is most valuable and most difficult. The mechanism: an element with aria-live is registered as a region; when its content changes, the browser fires an announcement event carrying the changed text and the politeness level. A polite region waits for a pause in speech; an assertive region interrupts.

Three failure modes are common and all of them are invisible in a static tree. The first is a region that is created at the same time as its content: a container inserted into the DOM already containing the message produces no change event, because the change was the insertion of a subtree, not a mutation inside a live region that already existed. The fix is to have the container present and empty before the message is written into it, and the test is a sequence: assert the container exists, then trigger, then assert the announcement. The second is a message that is written too fast after the region is registered, before the reader has attached its observer; a small deferral is the usual remedy. The third is a region whose content is replaced rather than appended, where some readers announce the whole region again and others announce only the difference.

The politeness level has a testable consequence. An assertive region used for a routine status message will interrupt whatever the user was reading, and a test can record the ordering: if the utterance for the status arrives between two halves of an unrelated announcement, the region is behaving assertively. Ordering is the property worth asserting, not the wording.

Making announcement tests maintainable

Three practices keep these tests from becoming a source of false failures.

Assert on the event, not the sentence. A test that checks an announcement event fired with the expected text, and separately checks the tree state that the reader will render, covers the page’s responsibility. The sentence the user hears is the reader’s responsibility, and pinning it means pinning a reader version.

Pin the reader when the test is about the reader. Some tests genuinely need a specific reader: whether a custom widget puts the reader into focus mode, whether a live region is announced at all, whether a table’s headers are associated in browse mode. These belong in a suite that records the reader, its version, and its verbosity settings, and that is run deliberately rather than on every commit.

Test the negative case. The most valuable announcement assertions are about absence: content that changes and produces no announcement, a status message that never reaches the user, a dialog that opens without its title being spoken. Absence assertions need a bounded wait and a recorded event log, which is why the event-stream capture point is worth the setup.

The deeper limitation is that speech is a serialization of a two-dimensional structure into a one-dimensional stream, and the reader chooses the serialization. The accessibility tree is the input; the utterance is one of several possible outputs; the judgement about whether the output conveys the page’s meaning is the part that needs a person listening.