Tag
test-collections
- Sep 11, 2026 Pooling Depth and the Unjudged Tail How pooled judgments are built from system submissions, why bias grows with pool depth and novelty, and how to measure the reliability of a partial pool. information retrieval
- Nov 24, 2025 How Relevance Judgments Are Encoded in a qrels File The four-field judgment record, the legacy iteration column, and why the choice of a binary or graded scale bounds what any later metric can express. information retrieval
- Oct 8, 2025 Judgment Drift and Assessor Agreement Relevance is not a stable property: label sets shift with the assessor, the interface, and the year. How kappa quantifies that disagreement and how to report metrics in light of it. information retrieval
- Jul 25, 2025 How Many Topics a Benchmark Needs Why the standard error of a mean metric is dominated by topic-to-topic variance, and how resampling shows when a query set is too small to separate two systems. information retrieval