From Invalidation to Present: The Frame Pipeline on a Mobile Device

A finger moves across a slider. The framework must read the touch, update a value, recompute the geometry that depends on it, record new drawing commands, and get those commands in front of the display in time for the next refresh. Each of those steps happens in a different component, some on different threads, and the whole sequence is paced by a hardware signal that the application does not control.

The pacing is the part that shapes everything else. A display refreshes on a fixed cadence: 60 Hz historically, 90 or 120 Hz on many current devices, with variable refresh rates that follow content. The compositor scans out a frame at each refresh, and a frame that is not ready when scanout begins is not shown until the following refresh. A pipeline stage that overruns by a fraction of the budget therefore costs not that fraction but a whole refresh interval, which is why frame-time distributions matter more than average frame time.

The vsync boundary and the two clocks

Every mobile graphics stack has a signal generated by the display controller at the start of each refresh, conventionally called vertical sync. The application’s rendering is synchronized to it, so that a frame produced by the application is presented at a predictable moment rather than tearing against a scanout in progress.

The synchronization is implemented as a callback from the display subsystem into the platform’s scheduling layer. On Android, the Choreographer receives the vsync signal and dispatches callbacks in a fixed order within the frame: input handling first, then animation, then the traversal that performs measure, layout, and draw. Applications and frameworks register work on this object rather than running their own timer, so that a callback is aligned with the refresh and so that work scheduled for the same frame is batched. Posting a frame callback is a request to run at the next vsync, and the framework coalesces multiple requests into one.

The ordering inside the callback is not arbitrary. Input is dispatched first so that a touch that arrived during the previous interval is reflected in the frame being built. Animation runs next so that animated values are advanced before layout reads them. Traversal runs last, with the full remaining budget, because it is the stage whose cost varies most with content.

Flutter separates the same work across threads instead of stages within one callback. The UI thread runs the framework’s build, layout, and paint, producing a layer tree; the raster thread consumes the layer tree and issues graphics commands; the platform thread handles input and the embedding. The vsync signal drives the UI thread’s frame, and the raster thread’s work for frame $n$ overlaps the UI thread’s work for frame $n+1$, which is the pipeline parallelism that makes a 16.6 millisecond budget achievable when each stage individually takes close to that.

What each stage can and cannot reuse

The value of the stage boundaries is that each one has a well-defined input and a cacheable output.

The build stage turns descriptions into a tree of retained objects. Its output depends on the state read and the parameters passed, so it can be skipped entirely for a subtree whose inputs are unchanged, as described in Recomposition Scopes: How Granularity Decides What Reruns. Skipping here is the cheapest possible saving because it removes all downstream work for that subtree as well.

Layout assigns sizes and positions. Its output depends on the incoming constraints, so it can be skipped for a subtree whose constraints are unchanged, which is what a relayout boundary enables; the protocol that makes this claim true is set out in Constraints Down, Sizes Up: Why Layout Is a Single Pass. Layout is the stage where the cost is least predictable: a text node that reflows can cost an order of magnitude more than a fixed-size box, and a nested intrinsic measurement can cost a full subtree walk per candidate width.

Paint records drawing commands into a display list. Its output depends on the geometry and the visual properties, so it can be skipped for a subtree that neither moved nor changed appearance. Paint is where a change of colour is cheap and a change of size is not, because a size change invalidates layout above and paint below.

Compositing assembles the display lists into layers and hands them to the GPU. Its output depends on the layer tree’s structure and the transforms between layers, so it can be skipped when only a transform or an opacity changed, and it is the stage where a scroll is handled: the content is already painted into a layer, and scrolling changes only the layer’s offset, which is why scrolling a static list costs almost nothing while scrolling a list that mounts new content costs the mounting.

Each stage’s cache key is the set of inputs that stage actually reads, and a change that invalidates an earlier stage necessarily invalidates all later ones for the affected subtree. A framework’s invalidation bookkeeping is therefore a bitmask per node, and the traversal walks the tree once, performing the earliest invalidated stage and continuing downward.

The deadline, and what overrunning it costs

The budget for a frame is the refresh interval minus the time the compositor needs. At 60 Hz the interval is 16.67 milliseconds; the application’s share is typically stated as something less, and at 120 Hz it halves to 8.33 milliseconds. The compositor’s own work, scanout setup, and the GPU’s execution of the submitted commands all consume part of the interval, so an application that uses the entire interval will miss frames even though its own stages each fit.

Missing the deadline has a specific and unintuitive consequence: the frame is not late by the overrun, it is late by a whole refresh interval, because the next opportunity to present is the following vsync. An overrun of one millisecond past the deadline costs 16.67 milliseconds of latency, and a sequence of frames that each overrun by a small amount produces a frame rate that is half the refresh rate rather than slightly below it. This quantization is why frame-time histograms of janky interfaces show spikes at multiples of the interval rather than a smooth distribution.

The standard mitigation is buffering. Double buffering lets the application render into one buffer while the compositor scans out another; triple buffering adds a third so that the application is not blocked waiting for the compositor to release a buffer. The cost is latency: a frame presented from a queue of depth three is at least one, often two, refresh intervals old by the time it is visible. Mobile platforms choose buffer depth per surface, and a game that needs minimal input latency will ask for a shallower queue than a scrolling list that can tolerate a frame of lag.

Where the measurement goes wrong

Frame timing is easy to measure incorrectly, and the errors are systematic.

Averaging frame times hides the distribution that matters. A run of 120 frames at 8 milliseconds and one at 40 milliseconds has an average below the 16.67 millisecond budget, and it also has a visible hitch. The useful statistic is the count of frames that exceeded the budget, or a high percentile of the frame-time distribution, not the mean.

Measuring on the UI thread only misses the raster thread. A frame whose UI work fits comfortably but whose raster work overruns will be presented late, and a profile that only instruments the build and layout stages will report a healthy frame. The stages have to be measured where they run.

Measuring in a debug build measures the debug build. Debug variants of most frameworks add assertions, disable inlining, skip compiler optimizations in the framework’s own code, and in some cases replace the rendering backend. Numbers from a debug build are useful for relative comparisons between two implementations of the same screen and useless as an absolute frame time.

Measuring without accounting for thermal state and refresh rate produces numbers that do not reproduce. A device that has been rendering continuously reduces its clock, and a device with a variable refresh rate changes the interval mid-measurement. Both effects are large enough to swamp the difference between two reasonable implementations.

The pipelining that a single-threaded model cannot have

The reason Flutter’s split between UI and raster threads matters is that a single-threaded pipeline serializes stages that could overlap. On a single thread, the total time for a frame is the sum of build, layout, paint, and rasterization. On two threads, the frame time is the maximum of the UI work and the raster work, once the pipeline is full, plus a constant offset for the handoff.

For a frame that spends 6 milliseconds on UI work and 10 on raster, the difference is 16 versus 10: the two-threaded pipeline meets a 16.67 millisecond budget with room to spare, and the single-threaded one does not. For a frame that spends 14 on UI and 3 on raster, the two-threaded pipeline is limited by the 14 and the single-threaded one by 17, so the benefit is smaller but still present. The benefit disappears when one stage dominates, which is why a heavy layout pass is not helped by moving rasterization off-thread and why the advice for such a pass is to reduce its cost rather than to parallelize it.

The cost of the split is synchronization. The UI thread produces a layer tree that the raster thread consumes, and the handoff must be atomic: the raster thread must see a complete tree, not a partially mutated one. Frameworks achieve this by treating the layer tree as immutable once submitted, with the UI thread building the next tree while the raster thread reads the previous one. That is the same copy-on-write discipline that snapshot state uses, applied to geometry, and it is why a frame’s layer tree can be retained and re-presented without re-running any of the earlier stages: an idle screen submits no new tree and the compositor re-scans the existing one, at a cost of essentially nothing.

What the pipeline cannot fix

A frame that misses its deadline because of a single expensive operation, a synchronous file read on the main thread, a large image decode, a first-use shader compilation, cannot be rescued by pipeline depth. The operation blocks the stage it runs in, and every subsequent stage for that frame waits. The remedies are all about moving the work off the critical path: decoding ahead of display, compiling shaders during a loading phase, reading files on a background thread and delivering the result as state. A related case sits in the input phase rather than the render phase: a gesture recognizer that waits too long before committing adds latency that no amount of pipeline depth removes, which is why arbitration resolves as early as the evidence allows, as described in Hit Testing and Gesture Arbitration Across Overlapping Views.

The other unfixable case is a stage that must run to completion for correctness. Layout of a newly mounted screen cannot be deferred, because the first frame must show the screen. This is why the first frame after navigation is the most expensive one in the pipeline, and why frameworks provide a way to prepare a screen’s work before it is shown, at the cost of doing that work whether or not the screen is ever displayed.