Hybrid Lexical and Dense Retrieval: Fusing Two Ranked Lists

Two retrieval systems can both be good and still fail on disjoint sets of queries. A lexical index matches terms and is defeated by vocabulary mismatch: a document that says “rate limiter” does not match a query that says “throttling”, even though they name the same thing. A dense encoder matches meaning and is defeated by rare exact strings: an error code, a part number, a function name, or a surname that the encoder has seen too few times to place reliably in the space. The failure sets barely overlap, which is the entire argument for running both and combining the results.

The argument is not that one is better. It is that the two methods make different assumptions and their errors are uncorrelated enough that a union recovers cases each misses alone. The practical difficulty is that combining them requires comparing quantities that are not comparable: BM25 produces an unbounded positive score with no absolute meaning, and cosine produces a bounded score whose distribution depends on the encoder and the corpus. Adding them directly, or weighting them by constants, mixes scales that shift with the query.

What BM25 scores, in one paragraph

BM25 ranks a document for a query by summing, over the query’s terms, a term-frequency factor damped by document length and weighted by inverse document frequency. The term-frequency factor saturates: the tenth occurrence of a term adds much less than the second, which is the difference between BM25 and a raw count. The length normalisation divides by a function of the document’s length relative to the average, so a long document does not win merely by containing more words. The two free parameters are usually left near their conventional defaults, one controlling saturation and one controlling length normalisation strength.

The important structural property is that BM25 scores are corpus-relative. The inverse document frequency of a term depends on how many documents contain it, so the same query against a corpus that has grown produces different scores. There is no calibration that makes a BM25 score of 12 mean “highly relevant” across corpora, and no reason to believe that a score of 12 in one index is comparable to a score of 12 in another.

Dense scores have the mirror problem. Cosine similarity is bounded and, for a well-trained encoder, clustered in a narrow band: most unrelated pairs land between 0.1 and 0.4, most relevant pairs between 0.5 and 0.9, and the band shifts with the encoder and the domain. A threshold of 0.7 is meaningful for one encoder and meaningless for another. Both score types are therefore ordinal within their own list and uninformative across lists.

Fusion by rank rather than by score

Reciprocal rank fusion sidesteps the calibration problem by discarding scores entirely. Each system contributes 1/(k + rank) for each document it returns, where rank is one-based and k is a constant conventionally set to 60, and the fused score is the sum over systems. Documents are then sorted by the sum.

The properties that follow from that formula are worth spelling out. A document at rank 1 in one list contributes 1/61, about 0.0164. A document at rank 1 in both lists contributes twice that. A document at rank 10 in one list contributes 1/70, about 0.0143, which is only 13 percent less than rank 1. The k constant is what produces that flatness: it compresses the difference between top ranks so that agreement between lists dominates position within a list. Lowering k sharpens the emphasis on top ranks; raising it flattens further.

The design intent is that consensus wins. A document that both systems place in their top ten outranks a document that one system places first and the other omits entirely, because the first document accumulates two contributions while the second accumulates one. That is the correct behaviour when the two systems’ errors are independent, and it is the wrong behaviour when one system is much stronger than the other, because a weak system’s agreement is then noise that can override a strong system’s confident first place.

def rrf(rank_lists, k=60, weights=None):
    # rank_lists: dict name -> list of doc ids, best first
    scores = {}
    for name, docs in rank_lists.items():
        w = 1.0 if weights is None else weights[name]
        for rank, doc in enumerate(docs, start=1):
            scores[doc] = scores.get(doc, 0.0) + w / (k + rank)
    # ties broken by doc id so the output is deterministic
    return sorted(scores.items(), key=lambda kv: (-kv[1], kv[0]))

The weights parameter is the escape hatch for the unequal-strength case, and it is also where the method stops being parameter-free. A weight of two for the dense list means its rank-1 document scores 2/61, enough to beat a document ranked first by lexical and second by dense. Choosing that weight requires evaluation data, which returns the calibration problem through a side door.

Alternatives to rank fusion, and what they cost

Score normalisation is the direct alternative: rescale each list’s scores into a common range, then add. Min-max normalisation maps the top score to 1 and the bottom of the returned list to 0, which makes the combination depend on how deep the list was cut; a list truncated at 100 has a different minimum than one truncated at 20, so the same document gets a different normalised score. Z-score normalisation assumes the score distribution is roughly Gaussian within a list, which is false for BM25’s long right tail. Both methods are sensitive to the depth of the candidate list, and depth is usually chosen for latency reasons rather than statistical ones.

Learned fusion trains a model to combine features from both lists, which can outperform any fixed rule and requires labelled relevance judgements for the target corpus. For a corpus with no judgements, the training data must come from somewhere else, and fusion weights learned on one domain transfer poorly to another. The engineering cost is a model in the serving path plus a training pipeline, against a rank-fusion rule that is eleven lines.

A third option is to use one system to filter and the other to rank. Run the lexical search to produce a candidate set of a few hundred documents, then embed and rerank those with the dense encoder, which avoids maintaining a full vector index. This is the classic recall-then-rerank arrangement, and it fails precisely on the queries the lexical stage misses: if a relevant document shares no terms with the query, it never enters the candidate set. The reverse arrangement, dense retrieval followed by lexical scoring of the shortlist, has the mirror blind spot. Neither is a fusion method; both are pipelines with one hard filter, and the fusion exists to avoid that filter.

Where hybrid retrieval still fails

Three failure modes survive fusion.

The first is a query with a single decisive token that both systems mis-weight. A query containing an exact identifier, where the identifier appears in ten thousand documents and the distinguishing term appears in three, will produce two lists that both rank by the wrong signal. Fusion combines two bad lists into a confident bad ranking. Mitigation lives in query analysis rather than in the fusion: detecting identifier-shaped tokens and requiring them, or boosting them, is a preprocessing step.

The second is duplicate content. If the corpus contains the same document in three formats, both systems return all three, and fusion rewards the duplication because each copy accumulates contributions. The fused top ten can then contain one document repeated five times and nothing else. Deduplication by content hash or by near-duplicate detection before fusion is necessary, and it is the same problem that overlap creates at the chunk level, discussed in Chunk Boundaries Decide What Can Be Retrieved.

The third is that fusion consumes the candidate budget. Two lists of 50 produce up to 100 candidates, and the prompt can hold perhaps eight chunks. Passing 100 candidates to a cross-encoder reranker costs 100 forward passes per query, which is often the dominant latency term in the pipeline; the trade between list depth, reranking cost, and final quality is treated in Cross-Encoder Reranking. Cutting list depth to control cost removes exactly the deep-rank documents that rank fusion is designed to rescue, since a document at rank 40 in one list and absent from the other is the case fusion handles well and truncation discards.

The lexical index is not free either

Running both systems doubles the index infrastructure: an inverted index with its own analyser, stemming rules, and stopword list, plus a vector index with its own build and maintenance path. The analyser is a source of subtle divergence. A pipeline that lowercases and stems for the lexical index but passes raw text to the encoder has two different views of the same corpus, and a query that matches after stemming may not match after tokenisation into subwords. Keeping the two views explicitly aligned, and documenting which normalisations apply where, prevents a class of bug that presents as inconsistent ranking rather than as an error.

The lexical index also carries the advantage that makes the pairing worthwhile in operational terms: it is exact, it is cheap to update, and it needs no accelerator. A term that appears in a newly added document is findable the moment the posting list is written, while the vector for that document may still be queued for encoding. During the window between the two, the hybrid system degrades to lexical-only for new material, which is a graceful failure rather than a gap. That asymmetry is a reason to keep the lexical path even when the dense path is the better ranker on average.