Semantics Trees and the Platform Accessibility Bridge

A screen reader does not read the view tree. It reads a tree of accessibility nodes supplied by the platform, whose shape is defined by a specification rather than by the drawing code. On Android that tree is built from AccessibilityNodeInfo objects that a view exposes through a provider; on iOS it is the set of accessibility elements that a view or view controller publishes; in a browser it is derived from the DOM plus ARIA attributes. The interface the user sees and the interface the screen reader traverses are two different structures over the same content, and keeping them consistent is a requirement that the declarative model makes harder, not easier.

Why the visual tree is the wrong tree

A visual hierarchy is organized for layout. A row of a list is a container with a background, a horizontal stack of an icon, a title, a subtitle, and a trailing chevron. That is four or five nested views for what a user perceives as one row with one label and one action. A screen reader walking the visual tree would announce the icon, the title, the subtitle, and the chevron as separate stops, requiring four swipes to pass a single row, and it would announce the container as an unlabelled group.

The accessibility tree is organized for traversal. It flattens the decorative and the structural into a single element carrying a label, a role, and possibly a value and a set of actions. The projection from one to the other therefore involves merging nodes that are visually distinct, splitting nodes that are visually one, reordering to match reading order rather than z-order, and suppressing nodes that carry no information.

Merging is the common case, and it is where most of the work is. Android’s AccessibilityNodeInfo has an explicit flag for suppressing a node: a view can be marked as not important for accessibility, which removes it from the tree entirely, so a decorative icon or a spacing view costs no traversal step. The container can separately be marked as a single focusable node for a screen reader, and the label announced for it is the one the application supplies, typically composed from the strings its children display. iOS does the same through isAccessibilityElement on a container, which makes the container the element and hides its subviews, with the container’s accessibilityLabel computed by the application.

The flag-based model has a specific failure mode. A container whose label is composed from its children will compose whatever the children currently are, so a row that conditionally shows a badge will announce the badge as part of its label when present and omit it when absent, without any code change at the container level. That is usually correct, and it becomes incorrect when the badge is decorative or when its text is a number that needs a unit to make sense. The remedy is at the child: a child marked as not important for accessibility contributes nothing to the label, so the container’s announcement stays stable across the condition.

Roles, states, and the mapping obligation

The tree carries more than labels. Each node has a role, and the role determines what the assistive technology offers: a button can be activated, a switch can be toggled, a slider can be adjusted, a heading can be used for navigation, a list announces its item count. The mapping from a framework’s widget to a platform role is a specification obligation, documented in the Core Accessibility API Mappings, and getting it wrong is not a matter of imperfect wording but of a control that cannot be operated at all.

A custom control drawn by the framework has no inherent role. A toggle implemented as a rounded rectangle that the application paints and animates is, to the platform, a plain view with a touch handler. Nothing about it says “switch”, so a screen reader announces an unlabelled element and offers no toggle action. The framework must attach the role and the state explicitly, and the state must be updated whenever the visual state changes; a switch whose accessibility value still says “off” after being turned on is worse than no switch, because the user is told the action had no effect.

The state surface is larger than it first appears. A node can be checked, selected, enabled, focusable, expanded, or busy, and each is a separate attribute with its own semantics. Android exposes these as properties on the node info object and requires the view to refresh them when they change, which in practice means implementing a callback that rebuilds the node description on demand rather than caching it. iOS exposes a trait set plus a value string, with the trait set carrying the role-like information and the value carrying the current state.

Declarative frameworks have an advantage here that is easy to waste. Because the widget description is rebuilt from state, the accessibility attributes can be derived from the same state in the same expression, so a switch’s checked attribute and its visual thumb position cannot disagree. Compose makes this explicit with a semantics modifier that takes the role, the state description, and the click action as parameters evaluated on each composition, and the semantics tree is regenerated along with the description. SwiftUI’s accessibility modifiers work the same way. The framework still has to be told; it cannot infer that a rounded rectangle is a switch.

The tree must be produced incrementally

A screen reader does not read a snapshot and stop. It holds a reference to the focused node, tracks the user’s traversal position, and expects to be notified when the tree changes. The platform’s contract is that a view reports structural changes as events, and the assistive technology updates its cached tree from those events rather than re-reading everything.

This is the hardest part of the bridge for a framework that rebuilds descriptions freely. A reconciler may replace a subtree in one pass, and the platform expects a sequence of events describing what was added and removed, in an order that leaves the tree consistent at every step. A framework that reports a wholesale invalidation forces the screen reader to rebuild its entire model, which loses the focus position and, on a large screen, produces a noticeable pause before speech resumes.

The practical consequence is that accessibility events are emitted from the same reconciliation pass that produces the visual mutations, not from a separate traversal. The reconciler already knows which nodes were inserted, removed, and moved; the accessibility bridge consumes that same diff and translates it into platform events. Frameworks that compute the accessibility tree lazily, on request, have to reconstruct the diff from a cached previous tree, which is why they tend to emit coarser events.

The focus position is the state that must survive the diff. If the focused row is removed, focus must move somewhere defined, and the platform’s rules for where it moves are not the same across platforms. A list that removes the focused row and does not announce a new focus leaves the screen reader’s cursor on a node that no longer exists, and the next gesture produces either silence or a jump to the top of the screen. The correct behaviour is to move focus to a neighbour and post an event announcing the move, and the framework cannot do that without knowing which node had focus, which means the bridge must track focus as part of its state rather than query it on demand.

Reading order and the geometry trap

Traversal order is defined by the tree, not by position on screen, and the two can disagree. A layout that places a floating action button at the bottom right of the screen, later in the view hierarchy than the content it overlaps, produces a traversal order in which the button comes after all the content. That is often right. A layout that places a header in a separate container above a scrollable list, but declares the list first and positions the header with a translation, produces a traversal order in which the header comes last while appearing first. That is wrong, and it is invisible to a sighted user.

Frameworks that compute geometry during layout can sort the accessibility tree by position, and some do as a fallback for content whose order is ambiguous. Sorting by position is a heuristic that fails for multi-column layouts, where the correct order is column by column rather than row by row, and for overlapping content, where the visually topmost element is not the first to be read. The reliable answer is an explicit ordering property, and the reason accessibility guidance keeps recommending a logical tree rather than a geometric one is that geometry is a poor proxy for reading order in exactly the cases where it matters.

The interaction with the hit-testing pass described in Hit Testing and Gesture Arbitration Across Overlapping Views is worth stating: the two trees can disagree about what is at a point, and the accessibility tree’s notion of an element’s frame is used by the platform for touch exploration. A merged node reports the union of its children’s frames, so exploring by touch highlights a region that is the bounding box of everything merged, which for an L-shaped composition includes area that belongs to no child. Splitting the node or adjusting its frame are the two remedies, and the choice depends on whether the merged node is genuinely one control or merely a visual grouping. Because the frame reported to assistive technology comes from the layout pass, a node whose geometry is stale by one frame is announced with the wrong bounds; the stages that produce and publish that geometry are traced in From Invalidation to Present: The Frame Pipeline on a Mobile Device.

Virtualized content and the tree’s size

A list with ten thousand items has ten thousand accessibility nodes if the tree mirrors the data. The platform cannot handle that, and neither can the assistive technology: a node tree of that size costs memory proportional to the item count on both sides of the bridge, and traversal becomes unusable.

The answer is that the accessibility tree follows the same virtualization as the visual tree. Only the materialized window of items appears, which keeps the tree’s size proportional to the viewport. That is correct for traversal but wrong for the platform’s model of a list, which expects to know the item count and the position of the focused item within it. The node must therefore carry the collection information separately: the row count, the index of each visible row, and an indication that the list is scrollable so that the assistive technology can offer a scroll action rather than requiring a swipe through every row.

The consequence is that a scroll performed by the assistive technology is a programmatic scroll, not a touch scroll, and the list must handle it as such: move the viewport, materialize the newly visible items, and post the corresponding tree changes. A list that only responds to touch gestures will appear to the screen reader as a fixed set of a dozen rows with no way to reach the rest, which is a complete loss of function for the content beyond the first screen.

What the bridge cannot be told after the fact

Accessibility metadata applied at the end of development is applied to a tree that was designed for a different purpose, and the two usual outcomes are over-labelling and under-labelling. Over-labelling comes from giving every visual node a label to silence warnings, which produces a tree with one stop per decorative element. Under-labelling comes from relying on the platform to infer text from a drawing, which it cannot do for anything the framework painted itself.

Both are avoided by treating the semantics of a component as part of its interface, written where the component is written, and derived from the same state as its appearance. The declaration of a custom control is the natural place to state its role, its label, its value, and its actions, and the fact that the declaration is re-evaluated on every change is precisely the property that keeps the accessibility tree consistent with the visual one. The cost is a second description per component, evaluated on every pass; the alternative is a tree that describes a different interface than the one on screen.