Profiling JIT Code and Runtimes That Manage Their Own Memory

A native profiler that works by walking the stack and symbolising addresses encounters two things it was not designed for in a managed runtime. The code is generated at run time into anonymous executable memory, so no file on disk describes it. The data is moved by a garbage collector, so an address recorded at one instant may refer to a different object at another. Neither is a bug in the runtime; both are consequences of designs that were chosen for reasons the profiler does not share.

Generated code lives outside every symbol table

When a JIT compiles a method, it allocates a region in a code cache; a mapped region marked executable; writes the machine code, and records a mapping from that region to the method it implements. That mapping exists only in the runtime’s own metadata. From the operating system’s point of view the region is anonymous, executable, and carries no name. /proc/<pid>/maps shows it as a path-less executable range, and a symboliser that resolves only against files has nothing to say about any address inside it.

The consequence is predictable: a profile of a heavily optimised managed program shows a large fraction of samples in unresolved frames, and the resolved remainder is the interpreter, the runtime’s runtime-internal C++ code (which does have symbols), and native libraries. The JIT-compiled code, which is where the time goes, is invisible. The percentage of unresolved samples is a direct measure of how much of the picture is missing, and it is the first number to check.

The fix requires the profiler to obtain the runtime’s code map. Runtimes expose it in several ways: an agent interface that the profiler loads into the process, a serviceability mechanism that streams metadata over a socket, or a perf-style interface where the runtime writes a /tmp/perf-<pid>.map file containing start size name lines for each compiled method. The perf convention exists precisely because anonymous executable memory needs an external name source, and it is consumed by the standard tooling that already understands the format.

# Resolve an address against a perf-map style code map.
def load_map(path):
    ranges = []
    with open(path) as fh:
        for line in fh:
            start, size, name = line.split(" ", 2)
            ranges.append((int(start, 16), int(start, 16) + int(size, 16),
                           name.strip()))
    ranges.sort()
    return ranges

def resolve(ranges, addr, hits):
    lo, hi = 0, len(ranges) - 1
    while lo <= hi:                       # last range with start <= addr
        mid = (lo + hi) // 2
        if ranges[mid][0] <= addr:
            lo = mid + 1
        else:
            hi = mid - 1
    if hi >= 0 and ranges[hi][1] > addr:
        hits[ranges[hi][2]] += 1

The map is append-only during execution, so a reader must tolerate a partially written final line by skipping it. Methods can also be deoptimised and their code regions freed and reused, so the map’s timestamps matter: an address resolved against a map state later than the sample can name the wrong method. Production profilers include the map’s generation or timestamp with each batch of samples for this reason.

Inlined Java looks nothing like inlined C

The inlining problem is worse than in a static compiler. A JIT may inline through several virtual call sites, devirtualise based on observed classes, and inlining decisions are made per call site and revised as new classes appear. A single compiled method body can contain the code of dozens of source methods, and it can be recompiled with a different set of inlined callees when the profile of receiver types changes.

A code map that names only the compiled root method therefore attributes the entire inlined body to the outermost method. Runtimes that support accurate profiling emit inline metadata alongside the code map: a table mapping ranges inside the method to the inlined methods, so a symboliser can reconstruct the chain. Without that table the profile is a function-level summary at best, and the most inlined; hence most optimised, hence potentially most interesting; code is hidden inside its host.

Recompilation adds a temporal dimension. The same source method may exist in three compiled versions during a run: interpreted, tier-1 compiled, and recompiled with more inlining. A profile aggregating over the whole run merges all three, and the merged node’s width is the sum of three different functions’ costs. Segmenting by phase, or recording the compilation identity with each sample, separates them; the phase selection problem is the same one described in representative workloads.

Safepoints, safepoint bias, and samples that cannot land

Managed runtimes need to stop threads at consistent points to perform collection, deoptimisation, and class loading. Those points are called safepoints, and threads poll a flag to reach them. A stack walk is only possible when a thread is at a safepoint or within a region where its frame layout is known, so many runtime-internal profilers record samples only at safepoints.

This produces safepoint bias, and its effect is specific. A loop that performs no call, no allocation, and no backedge poll cannot be interrupted; the profiler’s sample is attributed to the last safepoint poll the thread passed. Hot loops therefore appear under the method that contains the poll rather than under the operations they actually perform, and in extreme cases a profile shows samples only in a small number of poll sites. Aggressive JIT optimisation can remove polls from loops whose trip count is known, shifting the attribution further.

Async profilers avoid the bias by interrupting threads at arbitrary instruction boundaries using signals or the kernel’s performance events, then reconstructing the frame layout from the runtime’s metadata. This is the same mechanism described in sampling profilers, with the added requirement that the walker understand the runtime’s frame format, which is not the platform’s calling convention.

The heap does not hold still

Attributing allocation to code is straightforward if the profiler is told about each allocation as it happens, and this is what allocation-profiling interfaces do: they record the class and the allocating stack for each allocation, with a sampling probability to bound cost. JEP 331 defines such an interface for the JVM at a low overhead, and equivalent mechanisms exist in other runtimes under different names.

Attributing a live heap to code is harder, because a heap profiler must examine objects that already exist. The obstacles are object movement and precise layout. A collector that relocates objects changes their addresses, so a recorded address is stale unless the profiler either stops the world or uses a read barrier. The collector may also need to update the profiler’s own references, which means the profiler is inside the object graph it is measuring. Precise collection means the exact set of pointers is known, which makes walking tractable, but conservative or compacting collectors with forwarding pointers complicate the picture at exactly the moment the profiler wants to look.

Allocation hot spots are often not the same as retention hot spots. A method that allocates a million short-lived temporaries contributes nothing to the steady-state footprint, while a method that allocates ten long-lived caches may set the heap size. The two questions require two instruments: allocation profiling for the first, and heap walking with reachability for the second. Conflating them produces the common error of optimising a high-allocation path that never survives a collection.

Overhead budgets differ by an order of magnitude

Instrumentation inserted by the runtime; allocation events, method entry events, monitor contention events; is subject to the same cost scaling described in call counting. In a managed runtime the call rate is high enough that full method tracing is unusable for anything but a short diagnostic run: a program making tens of millions of calls per second pays the event cost tens of millions of times per second.

The three mechanisms sit in distinct cost bands. Sampled CPU profiling with an async sampler costs a fraction of a percent to a few percent. Sampled allocation profiling with a sampling interval of a few hundred kilobytes costs similar, because the sampling reduces the rate by orders of magnitude. Full tracing of method entry, allocation, and monitors costs a multiple of the program’s own time and changes the behaviour it measures; a fact the runtime cannot avoid, since the cost is inherent in the event rate.

What a managed profile cannot tell you

Two limits are structural rather than incidental.

First, runtime-internal time is not visible. Compilation, collection, and deoptimisation happen inside the runtime’s own code. A profile that samples only application threads reports the application’s own cost and none of the runtime’s. On a program whose collection pauses consume a noticeable fraction of wall-clock time, the CPU profile of application threads can look entirely healthy while the program is slow, because the time is being spent elsewhere with the application threads stopped.

Second, the correspondence between compiled code and source is approximate. Optimisations including escape analysis, scalar replacement, and lock elision remove operations that the source performs. A profile that shows no allocation at a site where the source allocates is correct: the compiler removed it. Reading such a profile as evidence of a bug in the measurement leads to chasing a difference that is the optimisation working as intended. The profile describes the emitted instructions, and the emitted instructions are not a translation of the source but a rewritten program that computes the same result.