Allocation Rate, Heap Growth, and Garbage Collection Pauses on a Frame Budget

A managed runtime hides the allocator and the collector, and the hiding is nearly complete until a frame arrives late. The pause that causes the lateness is not usually the result of a large heap. It is the result of a large allocation rate, and the two are independent: a process with a small heap that allocates furiously collects constantly, and a process with a large heap that allocates rarely collects almost never.

Rate, not size, sets the frequency

A generational collector divides the heap by object age. New objects go into a young generation, which is small by design, and objects that survive a collection are promoted to an older generation. Most collections are young-generation collections, which scan only the young generation and the roots that point into it. Their cost is proportional to the number of live objects in the young generation, not to the size of the heap.

The frequency is set by the allocation rate against the young generation’s capacity. If the young generation can hold C bytes and the app allocates R bytes per second, a young collection occurs roughly every C / R seconds while the app is allocating. Nothing about the total heap enters that ratio. An app that allocates 4 MB per second into an 8 MB young generation collects about twice a second; halving the allocation rate halves the frequency without changing the heap at all.

This is why the productive optimization is almost always to allocate less per frame rather than to grow the heap. Growing the young generation reduces frequency but increases the live set each collection must scan, so the total collection time per second can stay flat while the individual pauses grow longer. Longer pauses are worse for frames than more frequent shorter ones, because a frame only cares whether the pause crossed its deadline.

What a pause costs the frame

A young collection in a stop-the-world collector suspends every thread that might touch the heap, including the main thread. The suspension is what makes the pause visible: if the main thread was mid-layout when it was suspended, the frame’s remaining work shifts by the pause duration, and if the shift crosses the vsync deadline the frame is dropped.

The duration has several components. Root scanning visits thread stacks, which means the pause is partly proportional to the depth of the stacks at the moment of collection, not just to the number of live objects. Copying live objects to the survivor space costs a read and a write per object plus a pointer update per reference. Card marking or remembered-set scanning adds a cost proportional to cross-generational references, which is why an old object holding a reference to a young object makes every subsequent young collection slightly more expensive.

A concurrent collector removes the stop-the-world requirement for most of the marking work but not for the parts that must be atomic: initial marking, final marking, and any phase where the mutator must be paused to establish a consistent snapshot. Those phases are shorter but still bounded by root count and stack depth, and they occur at times the app does not choose.

Allocation sites that matter per frame

The allocation rate is the sum over all sites, and the sites that dominate are rarely the ones that look expensive. A per-frame allocation inside a draw or layout path is the worst kind, because it multiplies by the frame rate and lands inside the budget. Common examples are a String built by concatenation in a formatter, a boxed primitive in a collection that only holds primitives, an iterator object per loop, a lambda that captures a variable and therefore allocates a closure, and a temporary array sized from a variable.

The arithmetic is worth doing once. A 60 Hz app has 60 frames per second. A single 100-byte allocation per frame is 6 KB per second, which is negligible. A thousand of them per frame is 6 MB per second, which is enough to drive a young collection every second or two on a modest young generation. The difference between negligible and dominant is a factor of a thousand, and it is usually a factor of a thousand in the number of sites, not the size of each. Because a collection suspends the thread that was mid-frame, the pause lands directly against the deadline worked out in Frame Budget Arithmetic and the Real Deadline for Main Thread Work.

Strings deserve separate mention because their cost is superlinear in the number of concatenations. Repeated concatenation in a loop creates a new string and a new backing array per iteration, copying the accumulated content each time. A loop that builds a message from n pieces allocates n strings and copies a total of O(n²) bytes. A builder amortizes the copying to O(n) and allocates a small number of arrays.

Code shape that avoids the cost

// Per-frame: allocates a new array and a new iterator per call.
List<String> labelsFor(int[] ids) {
    List<String> out = new ArrayList<>();
    for (int id : ids) {
        out.add(Integer.toString(id));   // one String per element
    }
    return out;
}

// Per-frame with reuse: the array is filled, not reallocated.
void labelsFor(int[] ids, String[] scratch, StringBuilder sb) {
    sb.setLength(0);                     // reuse the buffer's storage
    for (int i = 0; i < ids.length; i++) {
        sb.append(ids[i]).append(',');
        scratch[i] = sb.toString();      // still one String per element
    }
}

The second version removes the list, its backing array, and the iterator, but not the strings, because a String is immutable and each distinct value needs its own object. Removing those requires changing the interface: a caller that formats directly into a buffer, or a lookup table of pre-built strings for a bounded set of values. The general rule is that avoiding allocation means avoiding the object, not avoiding the call that creates it.

Primitive collections are the other large win. A map from int to Object allocates a node per entry and boxes every key outside the small cached range, and iteration allocates an iterator. An open-addressed int-keyed table stores keys in a primitive array and allocates nothing per lookup. The trade is that it is a second data structure with its own invariants, and it must be maintained.

Measuring the rate honestly

Allocation profilers sample allocations rather than recording them, and the sample rate determines what can be seen. A profiler sampling every few kilobytes will attribute most of the rate to whichever site allocates in large chunks, and will undercount a site that allocates many small objects. Both matter, because the collector’s cost depends on object count as much as on bytes.

The useful derived metric is bytes and objects allocated per frame, computed by dividing a trace’s totals by the number of frames in the trace. A figure in the low kilobytes per frame is comfortable; a figure in the hundreds of kilobytes per frame means the collector is running on the frame budget’s schedule rather than the app’s.

Heap growth is a separate signal and answers a different question. A heap that grows without bound across repeated identical interactions indicates retention, not allocation rate: something holds a reference to objects that should be collectible. The distinction matters because the remedies are opposite. High allocation rate is fixed by allocating less; retention is fixed by breaking a reference, usually a listener, a cache, or a static field holding a context.

Interaction with memory pressure and process death

The collector and the platform’s memory management push in the same direction. A heap that grows makes the process a larger target for the killer described in Memory Pressure, Kill Decisions, and What an App Can Observe Before It Dies, and a process under memory pressure gets less room to work with, which makes collections more frequent. An app that keeps its live set small is both less likely to be killed and less likely to jank while running.

Trimming callbacks are the moment to drop caches that were kept for speed. A decoded image cache is the usual candidate, because it holds the largest bytes for the least computation. Dropping it costs a re-decode later and saves the process from being the one the killer selects. The ordering to avoid is a cache that is dropped on trim and immediately refilled by the next frame, which turns a trim into a burst of allocation at the worst possible time.

Ways the accounting goes wrong

A collection during a transition. Animations allocate, and a screen transition allocates for its new views while the old ones are still referenced. The peak allocation rate of a session often coincides with the transition into it, which is why entry animations jank and steady-state interaction does not.

Fragmentation mistaken for a leak. A heap can be large and mostly free while an allocation still fails, because no contiguous region of the needed size exists. Managed runtimes that compact avoid this; runtimes that do not compact can fail an allocation with a heap that has plenty of free bytes. The symptom is an out-of-memory error during a large array allocation in a process whose total heap is well under its limit.

Finalizers and cleaners. An object with a finalizer is not collected in the collection that finds it unreachable; it is queued for finalization and collected later, after at least one more cycle. A finalizer that allocates or blocks delays that second collection and holds the object’s entire retained graph alive in the meantime. The correct pattern is to avoid finalizers entirely and use explicit close or a reference-queue-based cleaner, which runs on a dedicated thread rather than on the collector’s path.

Reference processing during a pause. Weak and soft references are cleared during collection, and the queue they feed is drained on a thread the app supplies. A large number of reference objects makes the collection’s reference-processing phase longer, and the app’s queue thread must keep up or the references accumulate. A cache built on soft references therefore adds a per-collection cost proportional to its size, which is the opposite of what a cache is for.