Why Textual Renaming Fails and Syntax Trees Do Not

Renaming a symbol is the least controversial refactoring there is. It changes no values, no order of operations, no control flow. It also breaks code more often than any other mechanical edit, and the reason is that a rename is not a string substitution. It is a claim about binding: every occurrence being replaced must refer to the same declaration, and every occurrence referring to that declaration must be replaced.

Text search answers neither half of that claim. It finds occurrences by spelling and replaces all of them. The gap between spelling and binding is where renames fail.

The four ways a spelling occurs without binding

A name in a source file can appear in positions that have nothing to do with the declaration of interest. Each position defeats plain substitution in a different way.

Shadowing. An inner scope may declare a variable that already has the name the outer one is being renamed to. Replacing every occurrence of the old spelling then produces two declarations competing for one name, and the language resolves the collision by the shadowing rule rather than by intent.

def summarize(items):
    count = 0                      # rename target: count -> total
    for it in items:
        total = it.n               # unrelated local that already has the new name
        count += total
    return count

A blind substitution produces total = 0 at the top, then total = it.n; total += total inside the loop, then return total. Every name resolves, nothing raises, and the function now returns twice the last item’s count instead of the sum. A language with an unused-variable check catches part of this class; Python has none, and the arithmetic still type-checks in most languages that do have one.

The same failure appears wherever a nested scope can rebind a name: a loop body in Go or Java, a comprehension in Python, a let inside a block in Rust or JavaScript. The general form is that the substitution renames two declarations into one name, and the language resolves the ambiguity in favour of the inner one.

Comments and string literals. A comment mentioning the old name is documentation and should usually change. A string literal containing the old name may be a serialized field, a metric name, a SQL column, or a configuration key, and changing it is a behavior change. Text search cannot distinguish these, and the distinction matters: renaming a struct field and its JSON tag together is a wire-format change.

Macro bodies and code generation. A macro that expands to code mentioning the name will not be found by searching the source, because the name does not appear there. Conversely, a macro parameter that happens to share the name will be replaced even though it binds nothing.

Unrelated declarations in other files. Two packages may declare the same name for different purposes. A repository-wide substitution renames both, and if the second one is public, it breaks every external caller.

What a syntax tree adds

A parser produces a tree in which each identifier occurrence is a node, and name resolution annotates those nodes with the declaration they bind to. Once that annotation exists, the rename becomes a set operation on a set of nodes rather than a pattern match on text.

The distinction is between a token stream and a resolved tree:

Representation	Knows	Cannot know
Bytes	Positions of substrings	Whether an occurrence is code, comment, or string
Tokens	Lexical category, so strings and comments are separable	Which declaration an identifier binds to
Parse tree	Syntactic position, so a.b versus a is distinguishable	Which a a field access resolves to
Resolved tree	Binding of every identifier to a declaration	Types of computed expressions, without a type checker

The progression is a progression in cost as well as power. Lexing is linear in file size and needs no imports. Parsing needs a grammar for the language and, in C and C++, needs the preprocessor to run first. Resolution needs the whole compilation unit and, for cross-package renames, the whole program.

That cost gradient explains why editors offer two tiers of rename: a text-based one that works instantly and a language-server one that requires the project to be indexed. The instant one is correct often enough to be useful and wrong often enough to be dangerous, and the failure is silent.

Implementing a scoped rename

A minimal correct rename for a language with lexical scoping needs three pieces: a scope stack, a set of declarations per scope, and a resolution pass that records, for each identifier occurrence, which declaration it resolves to.

class Scope:
    def __init__(self, parent):
        self.parent, self.decls = parent, {}
    def declare(self, name, uid):
        self.decls[name] = uid
    def resolve(self, name):
        s = self
        while s is not None:
            if name in s.decls:
                return s.decls[name]
            s = s.parent
        return None

Every declaration gets a unique id at parse time. Resolution walks outward from the current scope and stops at the first match, which is exactly the shadowing rule most block-scoped languages use. Occurrences are recorded as (node, uid) pairs. The rename then rewrites every node whose recorded uid equals the target uid, and no other node, regardless of spelling.

The pass must also handle the cases where scope is not purely lexical. A global or nonlocal declaration redirects resolution to an enclosing or module scope. A with or catch clause declares a binding in the enclosing block rather than a nested one in some languages. A comprehension has its own scope in Python 3 but not in Python 2. Each of these is a rule the resolver must encode, and each is a place where a hand-rolled renamer diverges from the language’s actual semantics.

Two implementation details separate a prototype from a usable tool. The first is preserving source positions: a rewrite that reprints the whole file destroys formatting, comments, and line numbers, which makes the diff unreviewable and breaks any tooling keyed to positions. Editors solve this by patching only the byte ranges of the renamed tokens. The second is handling generated and vendored files, which must be excluded or the rename will be reverted by the next generator run.

Renaming across a repository

Single-file renames are a solved problem inside an editor. Repository-wide renames are where the resolution requirement becomes expensive, because a rename that crosses package boundaries needs the union of all compilation units resolved together.

Three strategies exist, and they trade precision for tractability.

Compile the whole repository as one unit, resolve, and rewrite. This is precise and requires the whole thing to build, which is often false for a repository with several languages or platform-specific code.

Resolve per package, then propagate. Rename inside the declaring package, then treat each external reference as a textual occurrence of an exported name. This is precise for the declaration and approximate for callers, and it works well when the exported name is distinctive.

Resolve nothing and rely on tests. Rename textually and let the suite catch mistakes. This is the strategy that produces the failure modes above, and it is defensible only when the name is unique across the repository and the suite covers every caller.

The tools that make the second strategy practical are the ones built on a real compiler front end. Clang’s LibTooling exposes an AST with resolved references, so a rewriter can enumerate exactly the declarations and references to change. Roslyn exposes the same for C# through a semantic model rather than a syntax tree alone. Both exist because the resolution step is the expensive part and the language’s own front end already does it.

The residual risk after a correct rename

A tree-based rename that resolves correctly still changes observations outside the source. Stack traces contain function names, so a crash reporter’s grouping changes. Debug symbol names change, which affects profiled data and log correlation. Reflection and serialization by name break, since the runtime looks up the old spelling. In a language with structural typing by name, such as TypeScript, renaming a property changes whether a type is assignable.

These are not failures of the rename; they are the boundary between a rename and a change in an interface. The distinction is worth keeping explicit, because a rename that crosses a serialization boundary is not behavior-preserving under any observation set that includes the bytes on the wire. That boundary is the subject of What Behavior-Preserving Change Cannot Preserve.

The mechanical follow-on, once renames are reliable, is that larger structural edits become reliable too. A rename is a rewrite over a resolved tree with a filter on one declaration. Widen the filter and the same machinery performs an extract, an inline, or a signature change across every call site, which is Repository-Scale Codemods and the Review Problem.