Monomorphic Inline Caches and the Guard That Makes Them Work

A property access in a dynamic language has a name fixed at compile time and a receiver of unknown layout. The naive answer, a hash lookup, is correct for any receiver. The fast answer is to bet that the receiver at this particular access site will nearly always have the same layout, record the bet in the instruction stream, and check it before using the result.

An inline cache is that record. It lives at the access site, one per site, and holds the layout it observed and the offset that layout assigns to the property name. Execution becomes a comparison against the cached layout, a branch on mismatch, and a load from a constant offset on match. The check and the load are independent, so on an in-order core they overlap; the whole sequence is a handful of instructions with no memory dependency beyond the object header and the field itself.

The name “monomorphic” describes the state of a site that has seen exactly one layout. That state is worth distinguishing because it permits the strongest code: the guard is a single comparison, the offset is a literal, and the optimizer can treat the load as having no side effects and a known result type.

The site, not the object, is the unit

The cache is attached to the access site because that is where the information is useful. An object does not know which code reads its fields; a site does not know which objects it will see. Caching on the object would help only repeated access to the same object, and repeated access to the same object is rare compared to repeated execution of the same access on different objects of the same kind.

Consider a function that reads a radius field from a parameter. Every call passes a different instance. Caching the layout on the instance would be useless, because each instance is touched once. Caching the layout at the site works, because every instance was built the same way and carries the same layout pointer, so the guard succeeds on every call after the first.

This is the same structure as a branch predictor: the site is the thing with a history, and the history is a distribution over the values that reach it. The difference is that the cache’s prediction is checked explicitly and the check is cheap because the predicted value is a pointer already in a register.

What the guard actually has to check

Layout identity is necessary and not sufficient. A compiled load that assumes layout L puts property x at offset 24 must establish several facts before that load is correct.

The receiver’s own layout must equal L. This is the primary check, and it subsumes the property’s existence, its offset, and its attributes. If the receiver has layout L, then it has x at offset 24, and no further name comparison is needed.

The receiver must not be a different kind of heap object that happens to share the layout pointer’s position. Engines handle this by allocating layout pointers in a way that makes the check meaningful for any object, or by tagging the layout with a type and checking the tag. A string, an array, and an ordinary object have different header layouts, and reading offset 24 of a string as though it were a property slot would produce a value from the string’s character data.

The prototype chain must be unchanged if the property is not an own property. A site that reads obj.x where x lives on the prototype needs the prototype’s layout too, or a check that the prototype’s layout has not acquired x as an own property. Engines handle the second case by keeping the prototype’s layout pointer in the cache and comparing it, or by storing a validity token that the prototype’s layout invalidates when it changes. The mechanics belong to Prototype Chain Guards and Validity Cells.

The property must not have become an accessor. If x was a data property when the cache was filled and has since been redefined with a getter, the offset no longer holds a value to load. The descriptor flags cover this when the descriptor changes, but a redefinition that keeps the same descriptor identity would slip past. Implementations avoid the case by treating accessor definition as a layout change.

The receiver must not be in dictionary mode. An object converted to a hash-table representation may keep a layout pointer for compatibility, but its values are no longer at the descriptor’s offsets. Engines either replace the layout on conversion or set a bit that the guard tests.

A single comparison against the layout covers the first, fourth, and fifth conditions if the layout is replaced on conversion, and the third if the prototype’s layout is included in the comparison. The second needs a tag check in some designs. The common case therefore costs one or two compares and one predicted branch.

Filling the cache without deoptimizing

The first execution of a site has no cached layout. The runtime performs the full lookup, records the layout it found and the offset, and stores them at the site. Subsequent executions take the fast path. In an interpreter this is a table entry indexed by site; in compiled code it is a patch of the instruction sequence, or a load from a per-site side table.

There is an important asymmetry between the interpreter and the compiler. The interpreter can update its cache entry on every execution with no cost beyond the store, because the entry is data. Compiled code that has baked the layout and offset into instructions cannot be updated in place without either patching those instructions or routing through a side table. Patching requires that the code be writable and that the patch be visible to other threads; a side table requires an extra load but no code modification. Engines use both, depending on whether the site is in code that has been optimized.

Filling from the interpreter is also where the site’s state is decided. The runtime records not only the layout but the fact that only one layout has been seen. That count is the input to the next decision: whether to keep the monomorphic form, extend to a small set, or give up.

Why one shape deserves its own code

The value of the monomorphic state is not only that the guard is cheap. It is that the optimizer can specialize the rest of the code around a known layout.

With a known layout, the type of the loaded value is known if the property has only ever held one type at that offset, which is tracked separately as the field’s type feedback. The optimizer can then unbox the value, keep it in a register, and inline arithmetic on it. A loop summing p.x over an array of points can keep the running total in a floating-point register and read each field with one load, instead of calling a generic add that dispatches on tag bits.

With a known layout, the object’s shape is also known to the allocator, so escape analysis can determine that an object does not outlive the function and can be replaced with registers. This optimization is only available when the layout is fixed, because an object whose layout may change cannot be flattened.

With a known layout, an access to a second property of the same object needs no second guard, because the first guard already established the layout. A sequence of loads from one receiver collapses to one check and several constant-offset loads.

None of this is available at a site that has seen two layouts. The optimizer must either duplicate the code for each layout, which is what a polymorphic inline cache does, or keep the generic path. The monomorphic case is where the payoff is largest and the machinery smallest, which is why engines are willing to spend effort keeping sites monomorphic, and why the shape discipline described in Hidden Classes and the Transition Tree Behind Property Writes has performance consequences that reach far beyond the object model.

The invalidation problem

A cache holding a layout and an offset is a claim about the state of the heap. The claim can become false, and the code must find out.

If the layout is immutable and every change produces a new layout, then a guard against the old layout fails automatically and the site re-fills. That is the clean case, and it is why descriptors are immutable with respect to their property set. The failing guard costs a branch misprediction and a slow-path call, and the site may then become polymorphic, but no incorrect value is produced.

The unclean case is a change that does not alter the layout pointer. Redefining a property with a getter on the prototype, adding a property to a prototype that a site reads through, or changing a field’s representation without changing the layout are all changes that leave the pointer intact. These are handled by a separate mechanism: the runtime records, for each such assumption, a dependency that is checked when the assumption is created and invalidated when the change occurs. In an interpreter this means re-validating the cache entry; in optimized code it means deoptimizing the function and discarding the compiled version.

The cost of that mechanism is the reason engines prefer to make layout changes produce new layouts. Every invalidation is a scan of the code that depended on the old state, and the scan is proportional to the number of dependent sites. Making the guard fail naturally converts a global invalidation into a local re-fill.

Where monomorphism ends

A site that sees a second layout is not immediately a problem. It can hold both, compare against each in turn, and remain fast as long as the number stays small. The threshold is a matter of engineering rather than principle: each additional case costs a compare and a branch, and past a handful the chain of compares is longer than the generic lookup it replaces.

The transition from one cached layout to a small set, and from that set to the uncached path, is the subject of Polymorphic and Megamorphic Inline Caches. The important point for the monomorphic case is that the fast path is not a different algorithm from the slow path; it is the same lookup with the result of the lookup hoisted into the instruction stream and checked. The correctness of the whole scheme rests on the check being complete, which is why the conditions above are enumerated rather than assumed.