Detecting Benchmark Contamination from the Model Side

A benchmark score is only meaningful if the items were unseen. When test questions, their answers, or close paraphrases of either appear in the training corpus, the model can score by retrieval rather than by reasoning, and the reported number measures the overlap between two datasets as much as it measures capability. The contamination is rarely deliberate and almost always invisible in the score itself, since a contaminated evaluation looks like a good evaluation with unusually strong results.

Detecting it is a set-intersection problem with a hard constraint: the training corpus is typically not available to the evaluator, and when it is, it is too large to search naively. The available methods therefore divide into those that inspect the corpus, those that probe the model’s behaviour, and those that reason about how the data was assembled.

Exact and near-duplicate matching against the corpus

The direct method is substring search of each test item in the training data. Exact matching is cheap with a suffix automaton or a rolling hash index, and it catches the case where the test set was copied verbatim into the corpus. Verbatim copying happens more than it should: question-answer pairs from public datasets get reposted on sites that scrapers collect, and a corpus built by crawling the web will contain them.

Exact matching misses paraphrase, and paraphrase is the more common form. Near-duplicate detection typically uses n-gram overlap: for each test item, compute the fraction of its n-grams that appear anywhere in the corpus, and flag items above a threshold. The choice of $n$ trades specificity against insensitivity to rewording. Short n-grams match by chance at high rates in a large corpus, so a 13-gram threshold is common for flagging document-level duplication, while shorter orders produce false positives on ordinary English phrasing. The threshold has to be calibrated against a control: run the same detector on items that are known to be absent, and set the cut so the control’s flag rate is near zero.

def ngram_set(tokens, n):
    return {tuple(tokens[i:i + n]) for i in range(len(tokens) - n + 1)}

def containment(test_tokens, corpus_index, n=13):
    """corpus_index: set of all n-grams in the training corpus."""
    grams = ngram_set(test_tokens, n)
    if not grams:
        return 0.0
    return len(grams & corpus_index) / len(grams)

Containment rather than Jaccard is the right measure here, because the test item is short relative to the corpus and the question is what fraction of the item was seen, not how symmetric the overlap is. A test question whose 13-grams all appear in one training document has been seen, regardless of how much other text surrounds it.

The weakness of n-gram matching is that it detects textual overlap, not information leakage. A test item can be contaminated by a training document that discusses the same fact in different words, and no n-gram threshold catches that. Conversely, a legitimate test item about a common topic will share n-grams with training text about that topic without the answer being present. The detector measures a proxy, and the proxy has both false positives and false negatives that cannot be resolved without reading the matches.

Behavioural probes

The second family asks the model to reveal its familiarity. The simplest probe is a comparison of loss on the test item against loss on a matched control item that is known to be absent but similar in style and topic. A model that has seen the test text assigns it lower per-token loss than the control, because memorized sequences are predictable. The difference is the signal, and it requires a well-constructed control set: matched on length, register, and domain, differing only in whether the text was in training.

Constructing that control is the difficulty. A control drawn from a different domain confounds memorization with domain familiarity. A control drawn from the same domain but written recently may be absent from training for a temporal reason rather than a random one, which correlates with content. The cleanest controls come from held-out splits of the same source distribution, when such splits exist.

A second probe perturbs the item in ways that preserve the answer and checks whether performance drops. Changing the names in a word problem, reordering multiple-choice options, or rephrasing the question while keeping the answer intact should leave a capable model’s accuracy roughly unchanged. A model that has memorized the item’s surface form will lose accuracy when the surface changes, because the memorized association between the specific question text and the answer no longer applies. The gap between original and perturbed accuracy is a direct behavioural estimate of how much of the score came from the surface form.

The perturbation method has a specific blind spot: it measures sensitivity to the perturbation, and a genuinely capable model can also be sensitive to a bad perturbation. Rewriting a question can change its difficulty, introduce ambiguity, or alter what counts as a correct answer. Validating the perturbation requires human review of the rewritten items, which costs annotation effort proportional to the set size, and the result is only as good as the rewrites. The review also produces a measurable artifact: the agreement rate between reviewers on whether a rewrite preserves the original meaning, which is the same inter-annotator agreement question that bounds any human-labeled evaluation and carries the same implication, that a rewrite set with low agreement cannot support a strong conclusion about contamination.

The memorization curve and what it implies

The empirical regularity behind contamination concerns is that memorization depends on duplication count and model scale. A sequence that appears once in a very large corpus is unlikely to be reproduced, while a sequence that appears many times is reproduced readily, and larger models memorize more at a fixed duplication count. This is a statement about extraction, established by prompting models to reproduce known training sequences, but it has a direct implication for evaluation: benchmark items that were reposted many times on the web are the ones most likely to be memorized, and they are also the ones most likely to be flagged by n-gram detection, since duplication is what the detector sees.

The two signals therefore agree in direction but not in magnitude. High duplication in the corpus makes both n-gram flagging and memorization likely; a single occurrence makes both unlikely. The interesting middle case is a test item that appears once in a corpus the model trained on for a single epoch. N-gram matching flags it. Whether the model can exploit it depends on scale and on how strongly the surrounding context cues the answer.

Provenance and the temporal argument

The cheapest defense against contamination is temporal: evaluate on items created after the training data was collected. A question written in a later year cannot be in an earlier corpus, and the argument holds without any corpus inspection. This is why benchmark versions carry publication dates and why evaluations on recent material are treated as more credible than evaluations on material from years earlier.

The argument is weaker than it looks. A training corpus has a collection cutoff, not a creation cutoff, and documents created before the cutoff can be published after it. A benchmark assembled from older sources and released recently can be fully contained in a corpus built from those sources. The date that matters is the date the underlying text entered the world, not the date the benchmark was packaged.

Provenance metadata helps where it exists. Benchmarks that document their item sources allow an evaluator to check whether those sources were in a corpus, which is a smaller search than checking the whole corpus. When the sources are public and identifiable, this is the most reliable check available, and it is a manual one: a list of source URLs against a list of crawled domains, rather than a string search.

What to do with a contaminated item

The options are removal, reporting, and reweighting. Removal reduces the set size and changes the composition, so a contaminated subset that is removed entirely can shift the benchmark’s difficulty in a direction that is not obvious. Reporting contamination rates alongside the score lets a reader discount the number without destroying the item pool. Reweighting downweights flagged items, which keeps them in the set while limiting their influence, and it requires a defensible weight.

The common practice is to report both the full-set score and the score on a clean subset, with the contamination rate stated. That pair is more informative than either number alone, since the difference between them bounds the effect. A large difference indicates that the headline number is substantially memorization; a small one indicates that contamination exists but does not drive the result.

Any of these choices interacts with the uncertainty of the score itself. Removing items narrows the evaluation set, which widens the confidence interval around the clean-subset number. A contamination correction that removes a fifth of the items and reports a corrected score with a much wider interval has traded a biased estimate for an imprecise one, and which is preferable depends on the size of the bias being corrected.