Mobile Computing · Mobile Performance

Frame Budget Arithmetic and the Real Deadline for Main Thread Work

Why the per-frame budget is smaller than the refresh period, how the pipeline splits it between CPU and GPU, and which stages can run in parallel with which.

A display refreshes at a fixed rate, and every refresh presents whatever the compositor has ready. The interval between refreshes is the frame period: about 16.7 ms at 60 Hz, 11.1 ms at 90 Hz, 8.3 ms at 120 Hz. That number is usually quoted as the frame budget, and the quotation is wrong in a way that matters. The budget available to the code that produces a frame is smaller than the period, by a fixed offset plus whatever the pipeline withholds.

Where the deadline comes from

Rendering is a pipeline, not a function. The application records drawing commands on one thread, a renderer translates them into GPU work, the GPU executes them, the compositor combines the app’s output with everything else on screen, and the display controller scans the result out. Each stage consumes the output of the previous one, and each has its own latency.

The pipeline is fed by a clock. On a vsync-driven system, the compositor wakes at each refresh boundary, takes the most recent completed buffer from each visible surface, composites them, and hands the result to the display. A buffer that arrives after the compositor has taken its snapshot is not in this frame; it waits for the next one. So the deadline for a frame intended for refresh N is not refresh N itself but the moment the compositor begins its work for refresh N, which is earlier by the composition time.

The application does not know that moment directly. What it gets is a callback scheduled at the start of the vsync interval, and its own work must finish, plus the render thread must finish translating and submitting, plus the GPU must finish executing, before composition starts. That chain is why the effective budget for main-thread work is the period minus the pipeline’s depth.

Splitting the period between threads

The useful mental model is two stages that can overlap across frames but not within one. Within frame N, main thread work must precede render thread work, which must precede GPU execution. Across frames, main thread work for frame N+1 can begin while the GPU is still executing frame N. That overlap is the entire reason the pipeline exists: it lets a 20 ms of total work per frame be sustained at 60 Hz, as long as no single stage exceeds the period.

The overlap only holds if there are enough buffers. With a single buffer, the app cannot start writing frame N+1 until the display has finished scanning frame N, which serializes everything and leaves no slack. With two, the app can write into one while the other is displayed, but cannot begin the frame after that until the first is released. With three, the app has a buffer available almost continuously. Each additional buffer buys overlap and costs latency, as worked through in Buffer Queue Depth and Added Display Latency.

For budgeting, the consequence is that the constraint is per-stage, not per-total. A frame whose main thread takes 12 ms and whose GPU takes 12 ms can sustain 60 Hz with three buffers, because the stages overlap; the same frame with one buffer cannot, because 24 ms of serialized work exceeds 16.7 ms. Profiling tools that report a single frame duration hide this, and a frame that misses its deadline while no single stage exceeds the period is a buffering problem rather than a slow-code problem.

The measured interval is not the budget

A tool that reports “frame took 18 ms” is usually measuring from the start of the main thread’s callback to the moment the frame was handed off, or from one vsync callback to the next. Neither is the budget. The first excludes the render thread and GPU; the second includes idle time waiting for the next vsync and so can never exceed the period even when frames are being dropped.

The quantity that predicts dropped frames is the interval between the vsync at which the frame’s work started and the vsync at which the frame was actually presented. If that interval is two periods, one refresh went out with the previous frame still on screen. That is a dropped frame regardless of how the individual stages look. Platforms expose this as a count of frames presented versus frames expected, and the ratio is the honest jank metric; single-frame durations are an input to it, not a substitute.

There is a second source of apparent jank that is not a dropped frame at all: a frame presented on time but with content that is one frame stale. When an app writes into a buffer that the compositor has already latched, the update appears one refresh later than the input that caused it. The frame count is perfect and the latency is worse. Distinguishing these requires timestamping input events and comparing against presentation timestamps, which is what frame-timeline tooling does.

Why the budget shrinks on high-refresh displays

Doubling the refresh rate halves the period, and it does not halve the work. The same view hierarchy needs the same measure and layout passes; the same scene needs the same draw calls. What changes is that the app now has 8.3 ms instead of 16.7 ms to do it, so a pipeline that was comfortably inside the budget at 60 Hz is over it at 120 Hz with no code change.

Platforms handle this by making the refresh rate variable. A display can run at 120 Hz when the content is animating and drop to 60 Hz or lower when it is static, and the compositor decides per frame based on whether anything is changing. For an app, this means the budget is not a constant but a property of the current frame: a frame that arrives late can cause the system to lower the rate, which increases the period, which makes the next frame easier. That feedback makes high-refresh jank harder to attribute than 60 Hz jank, because the frame rate moves in response to the app’s own behaviour.

Variable rate also changes what “on time” means. If the compositor may present at any multiple of a base period, a frame that misses the 8.3 ms deadline may still be presented at the next opportunity without a visible drop, as long as the content does not require a specific cadence. Animations do require one: an animation authored for 60 Hz that runs on a 120 Hz display either renders twice as many intermediate states or repeats states, and either choice is visible if the timing is not handled in terms of elapsed time rather than frame count.

Accounting for the constant costs

Every frame pays costs that are not proportional to the content. Submitting work to the GPU has a fixed cost per submission, and splitting a frame into many small submissions multiplies it. A surface that must be resized, reallocated, or revalidated pays a cost that has nothing to do with what it draws. A frame that touches a texture for the first time pays for the upload.

These costs are what makes the first frames of a screen more expensive than later ones, and they explain a common pattern: a screen that janks during its entry animation and is smooth once settled. The animation frames are competing with allocation and upload work that only happens once. Budgeting for that means either pre-warming the resources before the animation starts, or accepting that the first few frames of a transition have a larger cost and reducing what else happens in them.

The largest constant cost is often the one nobody counts: the composition of the app’s surface with everything else. If another app or a system overlay is also updating, the compositor’s work per frame grows, and the time available to the app before composition shrinks. The app’s own budget is therefore not a fixed 16.7 ms minus its pipeline depth, but that value minus whatever the system is doing. This is why jank often appears during system-level events and not during the app’s own stress test.

Reading a frame’s timeline correctly

A frame timeline lists the stages with timestamps: when the vsync callback fired, when the main thread finished, when the render thread finished, when the GPU completed, when the frame was presented. The gap between the vsync and the presentation timestamp, divided by the period, is the number of refreshes the frame consumed. That quotient is the only figure that should be compared against a target.

Two mistakes are common when reading such a timeline. The first is treating a long gap between the main thread finishing and the render thread starting as app time; it is usually the render thread waiting for a buffer, which is a depth problem. The second is treating a long gap between GPU completion and presentation as GPU time; it is usually the frame waiting for the next vsync because it finished early, which is the desired state.

The arithmetic that ties it together: if a frame’s work spans k periods and the app wants to present every period, then the pipeline must hold at least k frames in flight, and the added input latency is at least k periods. Reducing k by making the work faster and increasing the buffer count both help throughput; only the first helps latency. The section on Attributing Jank to a Specific Pipeline Stage covers how to find which stage is the one setting k.

References

  1. Choreographer reference — Android Developers
  2. CADisplayLink: QuartzCore reference — Apple Inc.
  3. Vulkan Specification: Presentation and swapchain — Khronos Group