Skip to content
AI.info

Generative AI

Sparse, Dense, and Hybrid Retrieval

Compare retrieval methods, fusion strategies, filters, and evaluation metrics for real query populations.

By the end you can

Comparison

Sparse and dense retrieval fail on different queries

One DPR checkpoint reaches 80.1 top-20 accuracy on Natural Questions. The same checkpoint reaches 25.4 on birthplace questions of the form Where was [E] born?, where BM25 reaches 75.2. That is 49.8 points absolute, running the other way. Sciavolino and colleagues measured both numbers in 2021. Nothing about the model changed between them. The queries did.

Hybrid systems are valuable when the query population contains both failure modes.

FigureComparison · 3 columns

Sparse lexical retrieval

Score documents using term matches and corpus statistics.

  • Strong for identifiers and exact phrases
  • Transparent term evidence
  • Vocabulary mismatch hurts recall
  • Works well with Boolean filters

Dense vector retrieval

Compare learned query and document representations.

  • Captures paraphrase and semantic relatedness
  • Supports multilingual transfer in some models
  • Can blur negation and rare distinctions
  • Requires compatible embedding versions

Hybrid retrieval

Fuse lexical and dense rankings before reranking.

  • Improves mixed query coverage
  • Needs score calibration or rank fusion
  • Adds index and tuning complexity
  • Still depends on corpus and filters

Example

Different queries expose different retrieval strengths

Any test set worth running should contain all of these patterns, and each of the first five has been measured on a public benchmark.

  • Exact identifier: “Error E1427” strongly favors lexical matching. EntityQuestions inverts the usual ordering. It is built from 24 Wikidata relations, up to 1,000 sampled facts each. On Natural Questions DPR scores 80.1 top-20 and BM25 64.5. On EntityQuestions DPR scores 49.7 and BM25 71.2. The paper's conclusion states it plainly: “In this study, we show that DPR significantly underperforms BM25 on EntityQuestions, a dataset of simple questions based on facts mined from Wikidata.”
  • Paraphrase: “How do I stop automatic renewal?” may benefit from dense semantics — the direction dense retrieval was built to win. On Natural Questions, the 2020 DPR paper reports top-20 passage retrieval accuracy of 78.4% for DPR against 59.1% for a tuned Lucene-BM25 (b=0.4, k1=0.9).
  • Negated policy: “Accounts that are not eligible” tests whether embeddings preserve a decisive distinction. NevIR measured exactly that in 2024, on 2,556 contrastive pairs (948 train / 225 dev / 1,383 test) whose two documents differ only by a negation. Random chance on the pairwise task is 25%. TF-IDF scored 2.0%, DPR 6.8%, SPLADEv2 ensemble-distill 8.0%, ColBERTv2 13.0% and ColBERTv1 19.7%. No bi-encoder, sparse or late-interaction model cleared chance. Only cross-encoders did — seven of them, from MonoT5-small (61M) at 27.7% through MonoT5-base at 34.9% to MonoT5-3B at 50.6%. On 10 randomly sampled test instances, all three human annotators ranked every query correctly. The abstract: “We find that most information retrieval models (including SOTA ones) do not consider negation, performing the same or worse than a random ranking.”
  • Rare domain term: A specialized acronym can be split or underrepresented in a general embedding model. The EntityQuestions relations show how large that effect gets. Who is [E]'s child? BM25 82.9 against DPR 19.2. Who is [E] married to? BM25 85.9 against DPR 35.6. Where was [E] born? BM25 75.2 against DPR 25.4.
  • Multilingual request: Query and document languages may require cross-lingual representations or translation, and which retriever wins depends on the language. MIRACL, published in 2023, covers 18 languages. Its abstract reports that “In total, we have gathered over 726k high-quality relevance judgments for 78k queries over Wikipedia in these languages, where all annotations have been performed by native speakers hired by our team.” On the dev set the two retrievers invert by language. French: BM25 0.183 against mDPR 0.435. Indonesian: BM25 0.449 against mDPR 0.272, in nDCG@10.
  • Metadata condition: Jurisdiction, date, tenant, and document status should filter candidates before generation.

Fusion combines rankings, not truths

Hybrid search can normalize scores, learn how to combine them, or use rank-based methods such as reciprocal rank fusion. Each choice changes how one retriever can rescue or overpower another.

Reciprocal rank fusion is a formula, published in 2009. A document's score is the sum of 1/(k + r(d)) over the input rankings, with k = 60 — a constant fixed during a pilot and never re-tuned. On LETOR 3's 583,850 document-query pairs, RRF reached MAP 0.6051 against Condorcet 0.5917, ListNet 0.5846, RankSVM 0.5737 and RankBoost 0.5622. Across the pilot and TREC runs it beat Condorcet, CombMNZ and the best individual system by 4% to 5% on average. The paper gives the reason it works: “RRF is simpler and more effective than Condorcet Fuse, while sharing the valuable property that it combines ranks without regard to the arbitrary scores returned by particular ranking methods.”

Score-normalising fusion can also work with no tuning at all. On the MIRACL dev set, BM25 averaged nDCG@10 0.385 and mDPR 0.418. The blend s = α·s_BM25 + (1−α)·s_mDPR — both scores first normalised to [0,1], α = 0.5 set without tuning — reached 0.566. Recall@100 went 0.772 for BM25, 0.790 for mDPR and 0.880 for the hybrid. The gain is real because the parents fail on different languages. That is what makes the French/Indonesian inversion a condition to check rather than a hedge to recite.

Fusion should be tuned on representative queries with protected evaluation. A higher combined score does not create authority or semantic support.

Case

Dense retrieval won in domain, and BM25 held up outside it

One table in the 2020 DPR paper carries both directions of the argument. On Natural Questions, DPR reaches 78.4% top-20 passage retrieval accuracy against 59.1% for a tuned Lucene-BM25 (b=0.4, k1=0.9) — the in-domain kind of result behind the paper's reported 9%–19% absolute band. On SQuAD the order reverses: BM25 68.8% against DPR 63.2%. The paper does not treat that as noise. It explains: “First, the annotators wrote questions after seeing the passage. As a result, there is a high lexical overlap between passages and questions, which gives BM25 a clear advantage.” The reversal is a property of how the dataset was written. A production corpus has properties like that too.

Out of domain, the ordering is not DPR's to keep. BEIR evaluated 10 retrieval systems zero-shot across 18 datasets. Measured as average nDCG@10 relative to BM25: DPR −47.7%, DeepCT −27.9%, SPARTA −20.3%, ANCE −7.4%, TAS-B −2.8%, ColBERT +2.5%, and BM25 with a cross-encoder reranker +11%. BM25+CE beat BM25 on 16 of 18 datasets; ColBERT on only 9 of 18. The abstract sums it up: “Our results show BM25 is a robust baseline and re-ranking and late-interaction based models on average achieve the best zero-shot performances, however, at high computational costs.”

Figure

The measured gaps run in both directions: DPR’s in-domain margin, and the eighteen out-of-domain collections where BM25 held its ground.

Visual

Filters and search order affect both quality and security

Apply access and lifecycle rules before candidates reach the model. A filter that runs after fusion has already allowed an out-of-scope record into a ranking, a cache, and a log. So identity and scope are resolved first, and sparse and dense search run inside the boundary they draw.

FigureProcess · 5 steps
  1. 1

    Resolve identity and scope

    Determine tenant, role, jurisdiction, time, and allowed collections.

  2. 2

    Apply hard filters

    Exclude inaccessible, retired, or out-of-scope records.

  3. 3

    Run sparse and dense search

    Retrieve complementary candidate sets.

  4. 4

    Fuse and deduplicate

    Combine rankings while grouping repeated sources and chunks.

  5. 5

    Return traceable candidates

    Preserve method, score, source, version, and filter decisions.

Key idea

Retrieval evaluation begins with a defensible relevance set

A labeled query-passage set can be incomplete. Several passages may support one answer, and annotators may miss relevant sources. Treat unjudged results carefully rather than labeling every unknown candidate as wrong.

BEIR measured the size of that hole. On TREC-COVID, Hole@10 — the share of a system's top-10 hits that annotators never judged — was 2.8% for docT5query and 6.4% for BM25, against 30.6% for DPR and 31.8% for TAS-B. The authors then annotated 980 additional query-document pairs by hand and rescored. docT5query's nDCG@10 moved from 0.713 to 0.714. ANCE went from 0.654 to 0.735 — from just below BM25 to 6.7 points above it — and ColBERT gained 5.8 points. The paper names the mechanism: “Such a lexical bias disfavours approaches that don’t rely on lexical matching, like dense retrieval methods, as retrieved hits without lexical overlap are automatically assumed to be irrelevant, even though the hits might be relevant for a query.”

Measure recall at the candidate budget, ranking metrics, no-answer behavior, latency, and important slices. Read failure examples to distinguish corpus gaps from ranking errors. And check how much of each system's top-k was judged at all before you compare the systems.

980 hand-added judgments took ANCE from 0.654 to 0.735 and docT5query from 0.713 to 0.714 — the label set was ranking the retrievers as much as the retrievers were ranking the corpus.

Steps

Run a retrieval bake-off

Compare methods under the same corpus, filters, and candidate budget. Build the slices out of failures that have already been measured elsewhere: negation pairs of the NevIR kind, entity questions of the EntityQuestions kind, languages that invert the way French and Indonesian do on MIRACL. Then fix the fusion constant before the release set is opened, the way k = 60 was fixed in a pilot and never re-tuned.

FigureProcess · 6 steps

  1. 1. Build query slices

    Include exact, paraphrase, negation, rare term, multilingual, and no-answer cases.

  2. 2. Label useful evidence

    Allow multiple supporting passages and record authority.

  3. 3. Fix candidate budgets

    Compare recall and latency at the same top-k limits.

  4. 4. Tune fusion safely

    Use development queries and lock a protected release set.

  5. 5. Inspect failures

    Separate missing corpus, filter, tokenization, embedding, and ranking errors.

  6. 6. Version the stack

    Release index, models, analyzers, filters, and fusion settings together.

Retrieval is a portfolio of signals and constraints

Sparse search preserves exact lexical evidence. Dense search supports semantic matching. Neither ordering survives a change of query population. 78.4% against 59.1% on Natural Questions. 68.8% against 63.2% the other way on SQuAD. DPR at −47.7% of BM25's average nDCG@10 across BEIR's 18 zero-shot datasets. DPR at 6.8% on NevIR, where random ranking scores 25%. Hybrid systems combine the two when the query distribution justifies the extra machinery: on MIRACL an untuned α = 0.5 blend reached 0.566 nDCG@10 where BM25 reached 0.385 and mDPR 0.418.

The next lesson adds reranking and context assembly. Candidate recall is necessary, but the generator still needs a compact, ordered evidence package. BEIR's strongest zero-shot line was BM25 with a cross-encoder reranker: +11%, and better than BM25 on 16 of 18 datasets, at high computational cost.

Key takeaways