The Structure of False Positives: Over-Approximation and Suppression Debt

A null-dereference finding on a line that cannot execute is not a defect in the analyzer’s parser. It is the visible result of a decision made when the analysis was designed: to be sound, the tool must consider every path the program could take according to its abstraction, and its abstraction does not know which paths are actually feasible. The report is wrong about the program and right about the abstraction.

Undecidability makes approximation mandatory

Rice’s theorem states that every non-trivial semantic property of programs is undecidable. “This pointer is null at this dereference” is a semantic property, and it is non-trivial: some programs have it and some do not. No analysis can be simultaneously sound (report every real instance), complete (report no non-instance), and terminating on all inputs.

A tool therefore picks a corner. A sound analysis over-approximates the set of reachable states, so it may report a state that no execution reaches; those reports are false positives by construction. An unsound analysis under-approximates and misses real bugs. A complete analysis reports only real bugs but cannot terminate on all programs or cannot handle all constructs. Most linters choose soundness for a narrow property and accept the resulting noise, because a missed bug is usually considered worse than an extra report.

This is not a temporary limitation of current tools. It is a theorem, and it means the goal of “zero false positives” is achievable only by giving up soundness or by restricting the language. Every practical improvement is a better trade, not an elimination.

Technical and effective false positives are different quantities

A technical false positive is a report for which no execution exhibits the bug. An effective false positive is a report a developer judges not worth acting on. The two overlap but are not the same. A technically true finding on code that is about to be deleted is an effective false positive. A technically false finding that points at a genuine design smell may be acted on anyway.

The distinction matters because the responses differ. Technical false positives are reduced by improving precision: path sensitivity, alias precision, better models. Effective false positives are reduced by improving presentation and prioritization: showing the path, grouping duplicates, ranking by recent changes, and letting teams opt out of rules that do not fit their idiom.

Tools that measure only the technical rate optimize the wrong number. A rule with a low technical false-positive rate can still be useless if its true findings are all in code nobody owns, and a rule with a higher technical rate can be valuable if its reports cluster on newly written code. The signal that matters is whether developers change their code in response.

The sources are few and identifiable

Most false positives trace back to one of a small set of abstractions.

Infeasible paths are the largest category. The analysis merges branches or explores a path whose condition cannot hold, and reports a state that is unreachable. The path-sensitivity budget discussed in Path Sensitivity and Branch Correlation is the direct control on this source.

Context merging is the second. A function called with a validated argument and with an unvalidated one gets a single summary that says the argument may be unvalidated, and the finding lands on the validated caller. Context sensitivity reduces it, at a cost that grows with the number of call sites.

Alias imprecision is the third. Two pointers are assumed to alias because the analysis cannot prove they do not, and a write through one is reported as corrupting the other. Field sensitivity and strong updates reduce it; allocation-site merging and weak updates cause it, as the constraint choices in May-Alias Analysis and Points-To Sets make explicit.

Missing models are the fourth. A call into a library or a runtime is treated as arbitrary, so any property the tool tracks is assumed to be destroyed. A logging call looks like it might retain the buffer; a string builder looks like it might mutate its input. Handwritten models for common APIs remove most of this, and the maintenance of those models is a permanent cost.

Unsupported language features are the fifth. A macro, a generated file, a domain-specific embedded language, or a reflection-heavy framework falls outside the parser’s model, and the tool falls back to a conservative answer. The report is usually accompanied by no explanation of the fallback, which is why it is so hard to triage.

# A suppression carries a rule, a scope, and ideally a reason.
# Syntax varies by tool; the structure does not.
<suppress rule="null-deref" scope="next-line" reason="checked by caller contract">

The reason field is the difference between a suppression that a future reader can evaluate and one that merely hides a warning. Tools that require it get better data about why rules fail; tools that do not get a growing set of unexplained directives.

Suppression is a loan against future analysis

Every accepted finding that is suppressed removes a true positive from the tool’s output, but it also removes the tool’s ability to notice a new instance of the same problem on that line. A file-level suppression is the extreme case: it disables the rule for all future code in the file, including code written by someone who never saw the original justification.

The accumulation has a name in practice: suppression debt. It is measurable in directives per thousand lines, in the fraction of files with file-level suppressions, and in the age of the suppressions that remain. A rising trend usually means the rule’s precision is below the team’s tolerance, not that the code is getting worse.

Several designs reduce the debt. Line-scoped suppressions are better than file-scoped ones because they expire when the line changes. Suppressions that require a reason and an owner can be audited. Suppressions that are themselves reported when the underlying finding disappears prevent dead directives from accumulating. And a baseline that records accepted findings separately from the source keeps the suppressions out of the code entirely, at the cost of a file that must be regenerated when lines move.

Prioritization is part of precision

Given that some false positives are unavoidable, the tool’s job is to make the true ones cheap to find. That means attaching evidence: the path that reaches the sink, the call chain that connects the source, the branch condition that was assumed. A finding with a path can be checked in seconds; a finding with only a line number requires reconstructing the analysis in the reviewer’s head.

Grouping is the second lever. Ten reports from one root cause should be one report with ten locations, not ten entries in a list. Deduplication by normalized message and enclosing symbol, together with a stable fingerprint as described in Diagnostic Spans and the Encoding Problem, keeps a finding identifiable across runs even as line numbers move.

The third lever is placement in the workflow. A finding that appears in a diff review, next to the change that introduced it, is evaluated in context. The same finding in a nightly report of ten thousand entries is not evaluated at all. This is a presentation decision, but it determines whether the analysis has any effect.

Silence has a meaning too

The opposite failure is quieter. An analysis that gives up on a function, a file, or a language feature and reports nothing is indistinguishable from one that analyzed the code and found it clean. Users read silence as approval. A tool that wants to be trusted must distinguish “no findings” from “not analyzed”, whether through coverage reporting, a per-file status, or an explicit marker on the constructs it skipped.

The same principle applies to timeouts. A path-sensitive check that exhausts its budget and returns no finding has not proved the code safe; it has proved nothing. Recording the budget exhaustion and surfacing it, at least in aggregate, is what keeps the tool’s negative results honest.