Measuring Debt Without Inventing Numbers

A dashboard can report that a codebase carries 4,200 hours of remediation work, that 6.3 percent of lines are duplicated, and that average cyclomatic complexity is 7.1. None of these numbers decides anything. The first is derived from per-rule effort constants that were assigned once and never validated; the second counts textual matches, including generated code and test fixtures; the third averages a statistic that is not meaningful to average, since a file with one 200-branch function and forty trivial ones reports the same mean as forty-one moderately complex ones.

The underlying problem is a category error. Technical debt was defined economically — a principal, a recurring interest, a horizon — and those quantities are not visible in source text. What is visible are properties that correlate with them under specific conditions. A useful inventory is built by knowing which proxy tracks which part of the economic model, and refusing to present the proxies as if they were the model.

What a proxy can and cannot tell you

Interest is a rate: extra cost per future change. No static analysis observes future changes. So every static metric is a proxy for one of two things: the probability that a component will be edited (exposure), or the cost multiplier applied when it is (friction). Interest is roughly their product, and conflating them is the most common mistake in debt tooling. A deeply tangled module that has not changed in three years has high friction and no exposure, and its contribution to the total is zero.

The proxies therefore pair up:

Quantity	Static proxies	What they miss
Exposure	change frequency, author count, defect history	unstarted work; seasonal change
Friction	size, nesting depth, fan-in, cycle membership	domain complexity that is irreducible
Propagation	transitive dependents, cycles, co-change pairs	runtime coupling through data
Verification cost	branch count, configuration flags, test duration	untested states that are never configured

Anything presented as a single “debt score” is a weighted sum of these, and the weights are judgments. That is acceptable if the weights are stated and the output is used to rank items for human review. It is not acceptable if the output is treated as a measurement of value, because then the ranking starts driving behaviour — and weighted sums are easy to game.

Cyclomatic complexity measures testability, not comprehension

McCabe’s 1976 measure counts the independent paths through a function’s control flow graph: for a connected graph, edges minus nodes plus two. It was proposed as a bound on the number of test cases needed to exercise a basis of paths through the function, and in that role it is useful. A function with a complexity of 15 has a basis of 15 independent paths, so 15 test cases cover them; the number is an upper bound rather than a minimum, and infeasible paths bring it down further.

Its use as a proxy for understandability is borrowed, and the loan has terms. Extract-method refactoring lowers the number without changing the total decision count in the file, so a complexity threshold can be satisfied while the code becomes harder to follow because control now jumps between five functions. Conversely, a linear sequence of twenty independent statements has complexity 1 and can be perfectly clear. Switch statements over an enum inflate the count even when every branch is a one-liner.

Two properties make cross-codebase comparison unsafe. The measure is not additive across functions in any meaningful way — summing it produces a number that mostly reflects how many decisions the file contains, which is a size measure under another name. And it is not comparable across languages, because the constructs that count as decisions differ: pattern matching, exception handling, and short-circuit operators are counted differently by different tools, and some tools count boolean operators inside a condition while others do not.

The defensible use is local and threshold-based: a function whose complexity exceeds a configured limit is a candidate for inspection, because it is likely to be under-tested relative to its branch count. The number is a pointer to a conversation, not a finding.

Churn is the only honest proxy for exposure

Version history records where work actually happened, which is the closest available stand-in for where work will happen. A file edited in most weeks of the last year will probably be edited again next month; a file untouched for two years probably will not. Change frequency per file, aggregated over a fixed window, is cheap to compute and hard to argue with.

Refinements matter. Absolute counts favour large files that are edited often in small ways, so relative churn — lines added plus deleted, divided by file size — is used to compare across sizes. Author count per file is a second dimension: a file edited by many people requires more coordination and carries more merge-conflict risk, since each concurrent edit to the same region is a potential conflict. Defect history, where a defect tracker is linked to commits, converts churn from a guess about future cost into evidence about past cost.

The failure mode of churn is that it measures activity, not value. Generated files, lockfiles, and vendored dependencies appear at the top of every churn ranking and are not candidates for refactoring. Renames break the history: git log --name-only labels each commit with the path that existed at that commit, so a renamed file’s history is split across two paths and neither holds all of it, and a file split into three appears as three new files with no past. Any inventory that ranks by churn has to filter generated paths and accept that renames distort the tail.

import collections, subprocess

log = subprocess.run(
    ["git", "log", "--since=12 months ago", "--pretty=format:", "--name-only"],
    capture_output=True, text=True, check=True,
).stdout

changes = collections.Counter(p for p in log.splitlines() if p.strip())
for path, n in changes.most_common(20):
    print(f"{n:4d}  {path}")

Ten lines that answer “where does work land” better than any static score, with no configuration and no thresholds.

Combining exposure with friction

The composite with the best ratio of insight to false positives is churn multiplied by a friction proxy, usually file size or complexity, ranked and truncated to a review list. The reasoning is that neither factor alone identifies debt: a large stable file is cheap because nobody touches it, and a small volatile file is cheap because each edit is quick. The product identifies the places where both conditions hold, which is where interest is actually being paid.

This is the hotspot heuristic, and its output has a specific meaning that is often overstated. It identifies where cost is being incurred, not what the design flaw is. A hotspot file may be volatile because it is badly structured, or because it genuinely implements the part of the domain that changes every week. The second case is not debt at all; it is the system doing its job. The ranking’s only legitimate use is to allocate inspection attention, and inspection is what separates the two cases.

A second composite is worth mentioning because it captures propagation rather than friction: the number of transitive dependents of a module, weighted by the module’s churn. A volatile module with two hundred dependents transmits its changes outward; a volatile leaf does not. Static import graphs give this directly in most languages, with the caveat that reflection, dependency injection configured at runtime, and dynamic imports hide edges, so the graph is a lower bound.

Why aggregate debt ratios do not travel

Tools that report a single percentage — remediation cost divided by estimated development cost, or duplicated lines as a fraction of total lines — invite comparison between repositories. The comparison is not valid. The effort constants behind remediation cost are per-rule estimates that differ by tool and version; the rule sets differ; the denominators include or exclude tests, generated code, and vendored libraries; and a repository with a large amount of well-factored code can score worse than a smaller, simpler one purely because it has more of everything.

Within a single repository over time, the same ratio is more defensible, with one condition: the rule set and configuration must be frozen. Otherwise the trend measures the tool’s update cadence. Even then, the ratio is susceptible to the cheapest available improvement, which is deletion. Deleting unused code improves every ratio and is often genuinely correct, which makes it a good action and a bad measurement.

The honest position is that aggregate ratios are reporting artefacts for people who are not going to read the code. They belong in a slide, clearly labelled as a rule-based estimate, and never in a decision about where to spend a quarter.

Designing an inventory around a decision

The discipline that keeps measurement honest is to start from the decision and work backward. If the decision is which three modules to stabilise before the next platform migration, the inventory needs exposure and propagation, not comprehension. If the decision is whether a rewrite of one subsystem is justified, the inventory needs the principal estimate and the interest estimate for that subsystem specifically, and a codebase-wide score contributes nothing.

A workable structure for the first kind of decision has four columns and no total: component, change frequency over a stated window, number of dependents, and a short note on what kind of change is landing there. The note is not decoration. It is what distinguishes a hotspot caused by structure from a hotspot caused by domain volatility, and no metric makes that distinction.

What the inventory should not contain is a forecast in hours for work nobody has scoped. The principal of a refactoring is unknown until someone has attempted enough of it to see the dependents, and publishing an estimate before that point creates a number that will be quoted long after its derivation is forgotten. The economic framing in Principal, Interest, and Upkeep in the Technical Debt Model is useful precisely because it makes the unknowns explicit rather than hiding them inside a constant.

The last constraint is maintenance cost. An inventory that must be regenerated by hand decays within a quarter; one that is computed from history on demand survives. Prefer the three-line query over the curated spreadsheet, and prefer no metric at all to a metric that nobody would act on.

Two uses exhaust most of what a ranking is good for. One is scheduling, which is mostly an ordering problem once the items and their dependency graph are known, as set out in Sequencing Repayment Under a Fixed Capacity. The other is the rare quantity that can be measured directly rather than proxied: build and test duration, which the build system already records and which charges every change, discussed in Feedback Latency as Compounding Interest.