Hot Code Detection with Invocation and Back-Edge Counters

Every adaptive runtime faces the same accounting problem. Compilation is expensive, execution counts are not known in advance, and the code that deserves compilation is a small, changing subset of everything loaded. The mechanism that answers “which subset” is a set of counters attached to program structure, and the arithmetic on those counters decides the entire optimization policy that follows.

Two Counters for Two Granularities

A counter on function entry alone answers the wrong question. A function called once that loops a million times is hot in execution terms but cold by entry count, and compiling it at the moment of its single entry would spend compilation effort before any evidence that the loop repeats. Conversely a small accessor called a hundred thousand times from scattered call sites is hot by entry count but each invocation does almost nothing, so compiling it may save less than the dispatch overhead it removes.

The structure that resolves this is a counter on the function entry, used to detect frequently called functions, plus a counter on each loop back edge, used to detect functions whose interior repeats. When either counter crosses its threshold the function is submitted for compilation. In HotSpot’s terms the method-entry counter and the back-edge counter are separate, and a back-edge overflow triggers compilation of the method containing the loop rather than a synthetic wrapper for the loop.

Back edges are the right place to count because a loop with one back edge executes its body once per back-edge traversal, and the bytecode format makes back edges identifiable statically: a branch whose target offset is less than or equal to the branch’s own offset closes a loop. The JVM specification defines the relevant instructions in terms of signed branch offsets, so a runtime can find loops without a full control-flow analysis by scanning for backward branches. Nested loops each get their own counter, and the inner one overflows first, which is usually the correct priority.

The consequence of counting back edges rather than loop iterations is that the count measures traversals of the branch that closes the loop, which equals the number of iterations that actually repeat. A loop that exits early trips its counter less often than its bytecode suggests, and a method whose cost is spread across several short loops adds to several counters rather than to one, so no single back-edge count represents the method’s total execution. That is why a back-edge overflow submits the whole method for compilation rather than proving the loop alone is hot.

// Counter arithmetic as a runtime performs it, expressed in Java-like pseudocode.
void onMethodEntry(Method m) {
    if (m.isCompiled()) return;               // nothing to count once code exists
    if (--m.entryCountdown <= 0) {            // decrementing avoids an increment and a compare
        m.entryCountdown = m.entryBackoff;    // reset before submitting, to avoid a storm
        compiler.submit(m, Tier.OPTIMIZED);
    }
}

The decrementing form matters. A counter that increments toward a threshold must load, add, compare, and store on every entry; a counter that decrements toward zero needs only a load, a conditional decrement, and a branch, and the branch is almost always not taken. On entry paths that run millions of times before compiling, the difference is a measurable share of interpreted execution. The reset-before-submit ordering is equally important: if the counter were reset after submission, every entry during compilation would re-submit the same method and flood the compiler queue.

Decay, Backoff, and the Shape of the Threshold

A single fixed threshold is a poor fit because hotness is not stationary. A method hot during startup may be cold for the rest of the process, and a method that becomes hot an hour in should not be penalized for its quiet early life.

Two mechanisms address this. Counter decay periodically halves counters, or subtracts a fixed amount, so that a method must sustain its rate to stay above the threshold. Decay turns a cumulative count into a moving estimate, at the cost of a periodic sweep over the counter table. The sweep’s cost is proportional to the number of methods tracked, which is why some runtimes decay lazily, adjusting a counter’s effective value from a timestamp when it is next read rather than walking the table.

Backoff raises the threshold after each compilation. A method that has already been compiled once and deoptimized should need more evidence before being compiled again, or a pathological method can cycle between compile and deoptimize indefinitely. Doubling the threshold after each event bounds the total compilation work spent on a single method to a geometric series, so even a method that never stabilizes consumes a bounded fraction of compiler capacity.

The combination produces a policy with three parameters, the initial threshold, the decay rate, and the backoff factor, and the tuning of those parameters is where most of the practical disagreement lives. A low threshold compiles early and wastes effort on code that never runs again; a high threshold leaves genuinely hot code interpreted for longer. The right value depends on the workload’s lifetime, which the runtime cannot know, so implementations either expose the parameters or pick values that favour long-running processes.

What the Counters Cannot See

Counters measure execution frequency, and frequency is only one of the inputs an optimizing compiler wants. They say nothing about which types flow through a call, which branches are taken, or how large the values are. Those facts come from a different mechanism, type feedback collected at call sites and branches, and the two systems interact: a method compiled on entry count alone may be compiled before its call sites have recorded enough type information to specialize anything, producing generic code that a later recompilation has to replace.

The counters are also blind to cost. A method that executes a hundred thousand cheap bytecodes and a method that executes a hundred thousand bytecodes each taking a slow path through dynamic dispatch cost very different amounts, and entry-count thresholds treat them identically. Some runtimes weight the count by an estimate of instruction cost or by the sampled stack, which is the approach described under sample-based profiling: a sampler observes where time actually goes, at the price of statistical noise rather than exact counts.

There is a structural problem with per-method counters as well: they attribute execution to the method that contains the loop, not to the caller whose data made the loop hot. A generic collection method may be hot only because one caller passes large arrays to it, and compiling the collection method alone leaves the caller interpreted. Inlining, discussed in speculative inlining and guard placement, is the mechanism that lets the compiler act on caller context, and it depends on the caller having been compiled with knowledge of its callee.

Instrumentation Cost and Its Alternatives

The counter itself is nearly free in the interpreted path if the interpreter already dispatches per instruction: one decrement and one conditional branch in the handler for method entry and in the handler for a backward branch. Runtimes that keep counters in the method metadata rather than in the bytecode stream pay an extra memory load per event, which is why some interpreters cache the countdown in a register-adjacent frame slot while the method is executing.

Once code is compiled, the counters are usually removed. A compiled method does not need to prove again that it is hot, and leaving the decrement in generated code would tax every call. The exception is a runtime that recompiles at higher tiers, where the compiled code carries a cheaper counter, often a loop iteration counter incremented in the loop preheader and tested once per iteration, to decide when to promote to the next tier.

On-stack replacement needs its own trigger, because a method that is already executing cannot wait for its next entry. The counter that fires OSR is attached to the loop rather than the method, and the transfer from interpreted loop to compiled loop requires reconstructing the frame state at the loop header, which is covered in tiered compilation and on-stack replacement.

Thresholds as a Policy Surface

The numbers chosen for these thresholds are visible in behaviour even when they are not documented. A runtime that compiles after a few thousand back-edge trips produces visible warmup: the first seconds of a process run slower, and short benchmarks measure the interpreter rather than the compiler. A runtime that waits for tens of thousands produces a longer warmup and better steady state, because the code it compiles is code that has proven itself. Neither choice is wrong; they optimize different lifetimes, and a runtime that serves both interactive tools and long-lived servers has to pick one default and let the other be configured.

The same counters that decide compilation also feed the recompilation policy. A method that has been compiled once and continues to accumulate count may be promoted; a method that deoptimizes frequently may be demoted or left at a lower tier permanently. The counter therefore does not just answer “compile or not” but participates in a state machine over compilation tiers, which is the subject of compilation budgets and recompilation policy.