Change Coupling Mined from Version History

Two files that always change in the same commit are coupled, whether or not either imports the other. The coupling might run through a shared database column, a configuration key, a duplicated constant, a protocol version, or a mental model that one developer holds and nobody has written down. Import analysis cannot see any of it. The commit history contains all of it, recorded as a side effect of ordinary work, and it takes a few hundred lines of script to turn that record into a list of pairs worth examining.

A commit is a set of files

The unit of analysis is the transaction: one commit, the set of paths it modifies. Co-change occurs when two paths appear in the same transaction more often than chance would explain. Nothing about the commit message or the diff content is needed for the first pass; the path set is enough.

The raw material comes from one git invocation per repository:

git log --since="12 months ago" --no-merges --name-only \
        --pretty=format:'%x01%H'

--no-merges removes merge commits, whose path sets are the union of two branches and would inflate coupling between everything either branch touched. The %x01 separator makes commits unambiguous to split, since path lines can contain almost anything except that byte.

From that, pair counts are straightforward, and the only care needed is cost. A commit touching k files produces k(k-1)/2 pairs, so a single commit touching 800 files produces about 320,000 pairs. Filtering oversized commits before counting is both faster and more accurate.

from collections import Counter, defaultdict

def pairs_of(commit_files, max_files=40):
    files = sorted(commit_files)
    if len(files) > max_files:      # mass rename or vendored import: not a coupling signal
        return []
    return [(a, b) for i, a in enumerate(files) for b in files[i + 1:]]

pair_counts = Counter()
file_counts = Counter()
for files in commits:               # commits: list of sets of paths
    pair_counts.update(pairs_of(files))
    for f in files:
        file_counts[f] += 1

A repository of 10,000 commits with a median of 4 files per commit yields tens of thousands of pairs, of which the vast majority occur once. The interesting pairs are the ones with a count in the dozens.

Confidence, support, and the popularity trap

Raw counts favour files that change constantly. The standard normalisation is a conditional probability: confidence of b given a is the number of commits containing both divided by the number of commits containing a. A confidence of 0.8 means that eight times in ten, changing a also changed b.

Thresholds that hold up in practice: at least five co-changes, and confidence in both directions above roughly 0.3. Requiring both directions matters, because a one-way relation is often a file that is a superset of another — a façade, a barrel file, or a dispatcher — rather than a mutual dependency.

Confidence alone still rewards popularity. A file that appears in 60 percent of all commits has high confidence with everything it touches, because everything touches it. The correction is lift: the observed co-change rate divided by the rate expected if the two files changed independently.

lift(a, b) = ( count(a and b) / N ) / ( (count(a)/N) * (count(b)/N) )

Lift near 1 means the pair changes together no more than chance predicts, given how often each changes. Lift well above 1 means a genuine association. Ranking by lift instead of by count moves the report from “the busiest files” to “the files that are secretly one thing”, which is the finding an evaluation wants.

There is one structural exception worth keeping rather than filtering: a source file and its test file will co-change with high lift and high confidence, and they should. Test-source pairs are expected coupling; excluding them by path convention reduces noise without losing signal, and leaving them in makes the report unreadable.

Churn, complexity, and where attention goes

Co-change finds pairs; a second measure finds where the pairs live. Hotspots combine change frequency with size or complexity, on the observation that effort and defects concentrate where code is both large and frequently modified. The simplest form ranks files by lines changed over the window multiplied by a complexity score such as cyclomatic complexity or a size measure in lines.

Neither factor alone is informative. A file with enormous complexity that has not changed in three years is stable and probably fine. A file that changes every day at forty lines is small and probably fine. The product of the two, computed over the same window as the co-change analysis, ranks files that are both hard to understand and constantly being adjusted.

The three measures compose into a useful working list:

• Hotspot with high co-change partners. A frequently edited, complex file that always changes with five others is a module boundary that has not been drawn. The cost of a change is already the cost of six changes, and the co-change is the evidence.
• Hotspot with no co-change partners. Frequently edited, complex, and isolated. This is either a file whose changes are genuinely self-contained, or one whose coupling to other files runs through data that the commits do not touch, such as a schema managed in another repository.
• Cold file with many partners. Rarely changed but always changed together with many others. Often a correct and stable protocol definition, and occasionally a hidden authority that everything depends on and nobody dares touch.

Cleaning history so it stops lying

History is a record of human workflow, and several workflows produce coupling that is not architectural.

Squash-and-merge versus atomic commits. A repository that squashes every pull request into one commit produces transactions that are actually entire features. Co-change within such a commit is a statement about one task, not about a recurring dependency. The signal survives if the window is long and the same grouping recurs, but it weakens. Conversely, a repository with very small atomic commits produces cleaner transactions and stronger signal.

Mass renames and reformats. A single commit that renames a package touches hundreds of paths and creates pairs with lift values driven by that one event. The max_files filter handles the extreme case; a second filter on commit size in lines handles the reformat that touches every file with a whitespace-only diff.

Generated code. Generated files change whenever their source changes, so they couple perfectly with their generators and their inputs and carry no architectural information. Excluding generated paths by convention is more reliable than any statistical filter, because the coupling is deterministic and total.

Repository boundaries. Analysis cannot see coupling across repositories unless the histories are combined, which is often impossible when the repositories are separately hosted. A change spanning two services in two repositories appears as two unrelated commits. This limits the method to a lower bound, exactly as import analysis is a lower bound for a different reason.

Branching model. Long-lived branches concentrate changes outside the mainline history until merge. A window that includes merged work sees the concentrated transaction; a window that misses unmerged branches misses the work entirely. Analysing the default branch over a fixed window is the consistent choice, stated with the analysis.

From pairs to a boundary decision

A co-change list is a diagnosis, not a fix. The finding that pricing/model.py and checkout/totals.py change together in 70 percent of commits has three possible readings, and only one of them is usually obvious.

The first is that the two files are one module with an artificial boundary. Merging them removes the ceremony of changing two files and makes the real boundary visible, which is the same conclusion the cycle analysis reaches from structure alone when the two are also mutually dependent.

The second is that a shared concept has been duplicated and drifts in step. The right answer is to extract the shared concept into one place that both depend on, which reduces the number of files per change rather than the number of modules.

The third is that the coupling is correct and invisible by nature: an event schema defined in one file and handled in another. In that case the pair belongs in the dependency rules enforced by the build, so the relation is documented and its removal is a deliberate decision rather than an accident.

Once a pair is understood, adding it to a watch list converts the analysis into a trend: the coupling rate for the pair should fall after the boundary is fixed, and if it does not, the fix was cosmetic. That measurement over time is cheap, since the extraction already exists, and it is the closest thing to a causal test available for a structural change.

The limits worth stating in the report

The method measures what was edited together, which is a proxy for what is coupled and an imperfect one. Three limits bound any conclusion drawn from it.

Frequency is not importance. The most-co-changed pair in a repository might be two files nobody cares about, because the work in that area is repetitive and low-value. Ranking by lift and complexity reduces this but does not remove it, and the report should be read alongside the quality-attribute scenarios that say what the system is for.

Co-change is not causation, and a pair that changes together may be two independent things that one process forces together, such as a deployment manifest and a version constant. That is real coupling of a kind, but its remedy is in the release process rather than in the module structure.

Small repositories and short windows produce noise. Under a few hundred commits, the counts are too small for lift to mean anything, and the honest output is “insufficient history” rather than a ranked list.