Bridging One Semantics Tree to Two Accessibility APIs

A screen reader does not read pixels. It queries an accessibility tree, which is a parallel structure maintained by the platform and consumed by the assistive technology. On iOS that tree is built from UIAccessibility properties and the accessibility elements an application publishes; on Android it is built from accessibility node information attached to views or supplied by a virtual tree provider. Both are trees of objects with roles, labels, values, states, actions, and parent-child relationships, and both are queried on demand rather than rendered.

A cross-platform toolkit that owns its rendering has no such tree. It has a scene graph of its own, and it must synthesise an accessibility tree for each platform from a description of what the widgets mean. That synthesis is the semantics bridge, and it is the part of a cross-platform architecture where abstraction costs the most, because the two target APIs do not share a vocabulary.

The two models are not the same shape

The first thing to establish is that the platforms do not merely name the same concepts differently. Their models differ structurally.

On iOS, an accessibility element has a set of traits (button, header, selected, adjustable) carried as a 64-bit option set, a label, a value, a hint, and a frame. Elements are arranged in a hierarchy, and a container can be an accessibility container with its own children. Custom actions are attached to elements. Focus movement is expressed by posting a notification that the screen reader should move to a specific element.

On Android, an accessibility node has a class name, a set of actions, text properties (text, content description, hint text), state properties (checked, selected, enabled, visible), and a collection of child nodes identified by integer identifiers. The virtual tree provider answers queries about nodes by identifier and must declare which node currently has input focus and which has accessibility focus, because the framework asks rather than infers.

The WAI-ARIA model, which is the reference vocabulary most shared design systems gravitate towards, is a third shape: roles, properties, and states, with a defined mapping to platform APIs for browsers. Using ARIA names as the shared vocabulary is reasonable, but the mapping from ARIA to either native API is incomplete, and the incompleteness is documented rather than accidental.

What is lost in each direction

The losses fall into four categories, and each has a distinct symptom.

Roles without a native equivalent. ARIA defines roles such as tablist, tab, switch, and progressbar. Android has node classes that correspond to several of these, and iOS has traits for some. A role with no equivalent must be approximated by the nearest one, and the approximation changes what the screen reader announces. A custom control announced as a plain button when it is a switch means the user is told to activate it rather than to toggle it, and the state is not announced at all.

Composite structures. A tab bar, a segmented control, or a data table is a composite: a container with children, where the container announces the count and position and the children announce their own state. iOS supports this with container traits and index properties; Android supports it with collection information on the node. A shared description that models the tab bar as a flat list of buttons loses the position announcement (“tab 2 of 5”), which is exactly the information a screen reader user needs to know where they are.

Actions. A widget may expose several actions beyond activation: increment and decrement for a stepper, dismiss for a chip, custom actions for a row. iOS attaches custom actions to an element; Android attaches action identifiers to a node. A shared description that carries only “activate” cannot express the rest, and the widget becomes unusable for anyone relying on the screen reader to reach the action.

Live updates. A change to a value that occurs without focus moving must be announced. iOS does this by posting a notification with a specific announcement string or by using a live region trait; Android does it with a live region property on the node. A shared description that updates its semantics only when the widget rebuilds will miss announcements that occur without a rebuild, which is the common case for a value that changes because of a timer or a network response.

Building the bridge from a semantics tree, not from widgets

The design that survives contact with both platforms is to make the shared layer produce an explicit semantics tree, separate from the widget tree, and to project that tree into each platform’s API.

The separation matters because the widget tree is optimised for drawing and the semantics tree is optimised for reading. A widget tree has nodes for backgrounds, dividers, and decorative images that must not appear in the accessibility tree at all. It has text split across several nodes that must be merged into one announcement. It has a card whose child texts should be read as a single sentence rather than as five separate elements. A 20-row list in which each row draws a background, a divider, an avatar, and three text runs contains 120 drawn nodes and 20 semantic ones, which is the ratio the projection has to get right. None of these transformations are drawing concerns.

// A semantics node is a platform-neutral description of what a widget means.
interface SemanticsNode {
  id: string;
  role: 'button' | 'header' | 'text' | 'image' | 'switch' | 'list' | 'listitem';
  label?: string;          // the accessible name
  value?: string;          // current value, for adjustable roles
  hint?: string;           // what activation will do, if not obvious
  states?: { checked?: boolean; disabled?: boolean; selected?: boolean };
  actions?: Array<'activate' | 'increment' | 'decrement' | 'dismiss'>;
  children?: SemanticsNode[];
  // Merged nodes are read as one element even though they draw as several.
  mergeDescendants?: boolean;
}

Two fields in that structure carry most of the weight. mergeDescendants corresponds to iOS’s container behaviour and Android’s approach of treating a view group as one node with a combined description; getting it wrong is the difference between a list row announced as one meaningful phrase and as five disconnected fragments. role is the only field that must map onto a finite vocabulary, and the vocabulary should be chosen to be the intersection of what both platforms can express, with explicit approximations for the rest.

Focus order is a tree property

Screen reader focus order follows the semantics tree’s order, not the drawing order. In a right-to-left locale, or in a layout where a visual arrangement differs from the logical reading order, the two orders diverge, and the semantics tree must carry the logical order; iOS expresses that order as integer indices, so a container with 5 children needs 5 distinct values in the sequence the reader should follow.

The common failure is a layout that positions children absolutely for visual effect. The drawing order is whatever the code appends, and the semantics tree inherits it, so the screen reader reads the price before the product name because the price was drawn first. The fix is to build the semantics tree from the logical structure and to let the visual order be whatever it is. That requires the shared layer to know the logical structure independently of the drawing, which is another argument for an explicit semantics tree.

Focus order also interacts with grouping. iOS exposes a property for the order in which elements are traversed, and Android derives order from the node hierarchy. A shared tree that groups children under a container gets the traversal for free; a flat tree requires the platform-specific ordering property, which then has to be computed per platform from the same logical order.

The accessibility tree must be updated, not rebuilt

A screen reader holds state: which element has focus, what was last announced, and the current scroll position in the tree. Replacing the whole tree on every frame destroys that state, and the symptom is a screen reader that jumps back to the top of the screen whenever any value changes.

Both platforms support incremental updates, and both require the application to describe them. Android’s virtual tree provider reports a content-changed event for a specific node identifier, so an update touches 1 node and a rebuild touches every node; iOS posts a layout-changed notification and relies on the element objects being stable. The bridge therefore needs stable identifiers across rebuilds, in the same way the reconciler needs stable keys, and it must translate a shared tree diff into per-platform update notifications.

The identifier discipline is the same one that governs view identity in Retained Native Widgets or an Owned Render Tree: an identifier must survive a rebuild when the widget is logically the same. A generated-per-build identifier turns every update into a full rebuild.

What the platform can do that a drawn widget cannot

Some accessibility behaviour is implemented by the platform’s own widgets and cannot be reproduced by a drawn one without explicit work. Three examples are worth naming because they are commonly missed.

Text selection and navigation within a text field: the platform’s text view exposes character-level navigation, selection, and the ability to read by word or character. A drawn text field that publishes only its full string as a value loses all of that. The bridge must either delegate to a hidden native text field, which is the standard technique, or publish the text with enough structure that the platform’s rotor can navigate it.

Adjustable controls: a slider or stepper must support increment and decrement actions, and the value must be announced as a percentage or a position within a range. A drawn control that supports only a drag gesture is unusable, because the gesture is not available to a screen reader user. Publishing the actions and the range is the only way to make it work.

The same asymmetry applies to the visual feedback a drawn control gives, which is a design-language question rather than an accessibility one: the press state, the focus ring, and the minimum target size are conventions the platform already implements for its own widgets. When a widget is drawn rather than delegated, those conventions become obligations, as described in One Description, Two Design Languages.

Tables and grids: position information (“row 3 of 12”) comes from collection properties that both platforms support and neither infers. A drawn grid must publish them explicitly.

Testing the projection

The projection can be tested without a screen reader, and doing so catches most regressions. Both platforms expose the accessibility tree to automation: the tree can be dumped and compared against an expected structure, with roles, labels, and states asserted per node. A test that asserts “the row announces the product name followed by the price as one element” fails when a refactor splits the row into separate nodes, which is exactly the kind of regression that a visual test cannot see.

The assertions worth writing are structural: node count, parent-child relationships, merged nodes, role per node, and the presence of actions. They are cheap to write once the semantics tree is explicit, because the expected structure is a literal of the same type the shared layer produces. A suite covering 20 screens with an expected tree per screen is a few hundred lines, and it fails on the day a refactor separates a label from the value it describes. And the assertions run on both platforms from one expected value, which makes them the one part of the accessibility work that is genuinely shared.

The limit of that testing is that it verifies the tree, not the experience. Whether the announcement is comprehensible, whether the grouping matches how a person would describe the screen, and whether the hint is useful are judgements no assertion captures. Those require listening to the screen reader, which remains the only reliable check, and which is why the semantics tree should be reviewed as prose rather than only as a data structure.