What Behavior-Preserving Change Cannot Preserve

Every refactoring is a claim that two programs agree on a set of observations, and the standard set is deliberately narrow. Return values, thrown exceptions, writes to durable media, and termination. Cost is not in it. Stack shape is not in it. Interleaving is not in it, and neither is the lifetime of a resource.

Those exclusions are what make the activity possible: if cost counted, no added indirection would ever be permitted, and structural change would be impossible in any language that charges for a call. But the exclusions are also where the technique meets its boundary, and a practitioner who does not know where the boundary is will eventually cross it while believing the change was inert.

The compiler is allowed to exploit what the source does not say

In languages with undefined behavior, the source does not fully determine the program, and a refactoring operates on the source. Two edits that are equivalent in the abstract machine can differ in what the compiler is permitted to assume.

Signed integer overflow is undefined in C. A compiler may therefore assume that x + 1 > x holds for a signed x and may delete a guard that tests it.

#include <limits.h>

/* Overflow here is undefined, so the compiler may assume it cannot happen. */
int add_wrap(int x, int y) { return x + y; }

/* Defined for every input: widen, then clamp. */
int add_clamped(int x, int y) {
    long long s = (long long)x + (long long)y;   /* cannot overflow */
    if (s > INT_MAX) return INT_MAX;
    if (s < INT_MIN) return INT_MIN;
    return (int)s;
}

The second version has defined behavior on overflow; the first has none, and the optimizer is entitled to reason as though overflow never occurs. That is not a refactoring, it is a bug fix, and a good one. The problem is that it will be described as a refactoring if it appears in a commit that also moves code around, and the description determines whether the new behavior is tested deliberately or discovered later.

The general form: any edit that changes which undefined behaviors the program relies on changes the set of programs the compiler may produce. Reordering floating-point operations changes results when the operations are not associative in the actual arithmetic, which is why -ffast-math is a semantic change rather than an optimization. Reassociating a sum in a loop is a refactoring in exact arithmetic and a behavior change in floating point.

Cost is excluded, and cost is sometimes the contract

Excluding cost keeps refactoring possible, and it also means a refactoring can violate a latency requirement while satisfying every test.

The mechanism is ordinary. Adding a call prevents inlining past the boundary. Extracting a function moves a loop body behind a call. Replacing a conditional with a virtual dispatch adds an indirect branch and defeats branch prediction. Introducing a parameter object adds an allocation on a path that previously had none.

None of these show up in a functional test. All of them show up in a system with a latency budget, and the failure is a distribution shift rather than an error: the median is fine and the tail is not.

The practical response is not to include cost in the observation set, which would make refactoring intractable. It is to record a performance baseline before a change that touches a hot path, so that the exclusion is a known risk rather than an unexamined assumption. Where no baseline exists, the honest position is that the change may have a cost effect and that nobody measured it.

There is a narrower case where cost genuinely is part of the contract: a program whose output includes timing, such as a benchmark harness or a real-time controller with a deadline. For those, the refactoring vocabulary does not apply and the change should be described as what it is.

Stack shape leaks through diagnostics

Extraction inserts a frame. Inlining removes one. Both preserve return values, and both change what a crash reporter records.

The leak has three paths. Stack traces change, so error grouping by frame changes and a monitoring system’s history is broken at the deploy. Debuggers and profilers attribute time to different functions, so a profile captured before the change cannot be compared with one captured after. And code that inspects its own stack deliberately, which is how some logging and authorization frameworks identify their caller, observes a different caller.

The third is the one that breaks correctness rather than reporting. A function that walks the stack to find the first frame outside its own package will find a different frame once an intermediate call is introduced, and if the result gates a permission check, the check now applies to the wrong context.

Languages make this better or worse. A language with inlined frames in its trace metadata can reconstruct the logical call stack after optimization; one without cannot. Either way, the refactoring’s claim about behavior was true only for the observation set that excluded stack inspection, and the system that inspects the stack was relying on an observation the definition does not protect.

Concurrency has no sequential observation

For sequential code, the observation set is a set of input-output pairs and the equivalence is a function equality. For concurrent code, the program’s behavior is the set of all executions consistent with the memory model, and a test observes one.

Two implementations can agree on every execution a test produces and differ on an execution the test never schedules. A refactoring that replaces a lock with a lock-free structure, or that narrows a critical section, changes the set of reachable interleavings. Every observed execution may still be correct, and the set may now contain one that is not.

This is not a testing gap that more tests close. The number of interleavings grows exponentially in the number of concurrent operations, and the observation set of any finite test run is a vanishing fraction of it. The tools that address it reason about the set rather than sampling it: model checkers that enumerate interleavings for a bounded model, and memory-model-aware race detectors that flag accesses not ordered by any synchronization.

The consequence for practice is that structural changes to concurrent code should be treated as behavior changes with respect to the interleaving observation, and verified by a method that reasons about schedules rather than by a suite that runs them. The order in which effects become visible to other threads is a property of the program, not of the code’s shape, which is the same distinction Lamport drew between the order of events in a system and the order in which they are observed.

Resource lifetime is behavior under pressure

A change that closes a file descriptor one statement later preserves every return value and changes the peak descriptor count. Under a limit of 1,024 open descriptors, a program that held 900 and now holds 1,100 fails, and the failure is an EMFILE at a call site that has nothing to do with the change.

The same shape appears with memory. Replacing a streaming read with a buffered one changes peak resident memory by the size of the buffer. Replacing a copy with a reference changes the lifetime of the underlying allocation, and in a garbage-collected language it changes the pressure on the collector, which changes pause behavior.

These are the observations that a test suite is worst at, because they are only visible at scale or under a limit. The mitigation is to make the limits explicit in the test environment: run the suite with a low descriptor limit, with a bounded heap, and with an allocation counter that fails the test when a hot path allocates. Each converts an invisible observation into a visible one.

When the refactoring silently fixes a bug

The most awkward boundary case is a change that preserves the intended behavior and not the actual one. A rewrite that happens to correct an off-by-one is a refactoring with respect to the specification and a behavior change with respect to the code.

The characterization suite catches this, which is exactly what it is for: the test fails, and the failure is a decision point rather than a regression. The decision has two valid answers. Either the fix is accepted and the test is updated in a separate, clearly labeled change, or the bug is preserved and the fix is deferred. Both are defensible; the indefensible outcome is a test updated in the same commit as a structural change, because the commit no longer has a single interpretation and the reviewer can no longer tell which diff lines were structural and which were corrective.

The discipline that resolves this is a commit-level one: one commit, one claim. A commit whose message says behavior is preserved and whose diff also fixes a rounding error is a commit where the claim is false and the fix is unreviewed as a fix.

A working boundary

The boundary can be stated as five checks made before an edit, and each one names an observation that the standard set omits.

The first check is whether the change alters which undefined or implementation-defined behaviors the program depends on. If it does, the change is a fix or a portability change, not a refactoring.

The second is whether the path has a latency, memory, or descriptor budget. If it does, a baseline is needed before the change, because the exclusion of cost is an assumption rather than a fact.

The third is whether the code is concurrent. If it is, the equivalence claim covers only the schedules that were run, and the verification method should reflect that.

The fourth is whether anything downstream reads names, stacks, or iteration order. If something does, those are observations in use even though the definition excludes them, and the change is a change to a de facto interface.

The fifth is whether the rewrite alters behavior that some caller depends on but no test asserts. That is the ordinary sampling gap, and the only defenses are the seam-level capture of Characterization Tests as a Refactoring Safety Net and the per-site reasoning that a codemod cannot perform, as described in Repository-Scale Codemods and the Review Problem.

A change that passes all five is a refactoring under the standard definition and under the practical one. A change that fails any of them is still worth making; it simply needs the verification appropriate to the observation it alters, and it needs a description that does not claim more than was checked.