Contract Tests for Stateful ARIA Widgets

A disclosure button, a tab list, a combobox, and a grid share a property that makes them testable in a way that a page does not: each has a documented contract. The authoring practices specify, per pattern, which keys do what, which states exist, which element owns which, and what should be announced when a state changes. The contract is written for implementers, which means it is written precisely enough to be turned into assertions.

The work of testing a widget is the work of transcribing the contract into the vocabulary of the accessibility tree and the browser’s key handling, then deciding which clauses survive contact with real readers.

The shape of a widget contract

Every pattern in the authoring practices has the same five sections, and each maps to a different test type.

The role and its required properties come first: what the element’s role is, which states it must carry, which it may carry, and what its accessible name comes from. These are assertions about the tree after the widget renders, and they are cheap.

Keyboard interaction comes second: a table of keys and the effect each produces. This is the largest section and the one that produces the most defects, because the keys that matter are the ones a mouse user never presses: arrow keys within a composite widget, Home and End for the first and last item, Enter and Space for activation, Escape for dismissal, and for some patterns the typeahead behaviour of printable characters.

Focus management comes third: where focus goes when the widget opens, when it closes, when an item is removed, and when the composite is entered with Tab. These are assertions about document.activeElement at defined points in a sequence.

States and properties come fourth: the attributes that change and when. A tab’s aria-selected is true for exactly one tab in a tab list. A tree item’s aria-expanded is present only if it has children. A combobox’s aria-expanded tracks its popup.

Announcements come fifth, and are the least specified: which state changes should be spoken. The practices describe expected behaviour in prose, and the actual utterance is the reader’s.

Turning the key table into a test

The key table is a matrix: rows are keys, columns are the widget’s possible states, cells are effects. A test is a walk of a subset of that matrix, asserting the effect after each key.

// One row of the tab list contract: Right Arrow moves to the next tab and selects it.
await page.getByRole('tab', { name: 'Overview' }).focus();
await page.keyboard.press('ArrowRight');
const selected = await page.getByRole('tab', { selected: true }).getAttribute('id');
expect(selected).toBe('tab-details');
expect(await page.evaluate(() => document.activeElement.id)).toBe('tab-details');

The assertions are deliberately doubled: the state changed and focus moved. A tab list that changes the selected tab without moving focus leaves the reader’s cursor on a tab that is no longer current, and a test that checks only the state passes while the user is lost. This pairing of state and focus is the general form for every composite widget.

The matrix has boundary cells that are more valuable than the interior ones. Arrow Right on the last tab must wrap to the first, or must stop, depending on the pattern; the practices specify wrapping for tab lists. Home and End must move to the first and last regardless of the current position. An arrow key pressed on a widget with a single item must not fail or lose focus. These cells are where implementations built by hand tend to be incomplete, because the happy path was tested and the edges were not.

Activation keys differ by role in a way that is easy to get wrong. A button activates on Enter and on Space. A link activates on Enter only; Space scrolls the page. A checkbox toggles on Space. A radio in a radio group moves and selects with the arrow keys, so a test that presses only Space covers none of the behaviour the pattern requires. A menu item activates on Enter and typically closes the menu. Each of these is a distinct assertion, and a widget that activates on both keys when it should activate on one is not merely over-permissive: pressing Space on a focused link-shaped control that responds to Space prevents the page from scrolling, which is a defect in its own right.

Ownership and the composite structure

Composite widgets are trees in the accessibility layer, and the structure is part of the contract. A tab list owns its tabs; a grid owns rows, which own grid cells; a listbox owns options; a tree owns tree items arranged in groups. The ownership relation is what allows a reader to report position within the widget (“tab 2 of 5”) and what makes arrow-key navigation meaningful.

Structure assertions are tree assertions. The count of owned children must match the count of items, and the index of the focused item must be derivable. aria-posinset and aria-setsize exist for the case where the DOM does not contain the whole set, as in a virtualized list where only the visible items are rendered, and a test of a virtualized list must assert those values rather than counting nodes, because counting nodes reports the window size rather than the set size.

The aria-owns relation is the case where the tree structure differs from the DOM structure, and it needs a test of its own. A combobox whose popup is rendered in a portal at the end of the body must declare ownership, or the reader will not treat the popup as belonging to the combobox. The assertion is not that the attribute exists; it is that the tree shows the popup as a child of the combobox. This is exactly the kind of assertion that a tree-based oracle makes possible and a markup query does not.

Dialogs as a contract with the browser

The dialog pattern is partly specified by HTML rather than by ARIA, and the split matters for testing. A <dialog> element opened with showModal() gets, from the browser: focus moved into the dialog, focus containment, Escape to close, and the rest of the document made inert. A dialog built from a div with role="dialog" and aria-modal="true" gets none of these from the browser; every one of them must be implemented in script, and every one is a place to fail.

The contract test therefore branches on the mechanism. For a native dialog, the assertions verify that the browser’s behaviour was not defeated: that showModal was used rather than show, since a non-modal dialog has no containment; that the dialog has an accessible name from aria-labelledby or aria-label, since the browser does not supply one; and that focus returns to the opener on close, which the browser does not do.

For a scripted dialog, the assertions cover the full list: initial focus inside, Tab and Shift-Tab both contained, Escape closes, background inert or aria-hidden, focus restored. The Shift-Tab direction is the one most often missing, and it is the one the focus protocol records as a distinct failure.

There is a third mechanism, the native <dialog> with show() plus a manual backdrop, which gets no containment and is a common source of a dialog that a keyboard user can tab out of. The test cannot distinguish the intent from the mechanism, so the mechanism has to be visible in the test’s setup: which method was called is a fact the test knows.

Where the contract runs out

The authoring practices are guidance, not a standard, and their status matters when a test fails. A pattern’s key table describes one recommended implementation. A widget that satisfies the role’s required states and is fully operable by keyboard can be conforming while differing from the table. Treating the table as normative produces failures that are not defects.

The announcement clauses are the least testable. “The state change should be announced” is not a statement about an event the page emits; it is a statement about what a reader does with the event. The page’s responsibility ends at firing the right event with the right state, and the question of what the reader says is a separate test with a separate stability profile.

The third limit is state space. A widget’s contract describes states, and the number of reachable combinations grows quickly: a combobox that can be collapsed, expanded with no options, expanded with options, expanded with a filtered list, and expanded with a highlighted option has five states before any content variation. A contract test suite covers the transitions the practices name and the boundaries those transitions have, and it does not cover the combinations. That is a sampling decision, and it should be recorded as one, because the claim the suite supports is “these transitions behave as specified”, not “the widget is correct”.

What the suite buys is the property that makes widgets maintainable: the contract is written down in executable form, so a refactor that changes the internal markup but preserves the role, the states, and the key behaviour passes unchanged, and a change that breaks a boundary cell fails with the cell’s name in the message. That is the same argument as for any contract test, and it holds here because the contract happens to be published.