What is hybrid search, and how do you merge BM25 and vector results?

In this article

1. Before you start
2. Why interviewers ask this
3. Hybrid search from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

Why interviewers ask this

A candidate who answers "use embeddings" to every search problem usually fails the first follow-up: what happens when the user types an exact error code or a product name? Hybrid search questions test whether you know that one retriever is not enough and that merging two retrievers is a design problem, not a formality. The usual form is a design prompt: build a search system that combines semantic and keyword retrieval over millions of documents under a latency target; the machine learning system design round is where it comes up.

A weak answer names the two retrievers and stops. A strong answer says why each one fails, how the lists are merged without comparing raw scores, what the merge costs, and how you would measure whether it helped, which links to large language model (LLM) observability. The rest of this page builds that answer from zero, then shows the arithmetic on a small example you can redo by hand.

Hybrid search from first principles

Hybrid search

Hybrid search is a retrieval method that sends one query to both a keyword index and a vector index and merges their ranked results into a single list, so that exact-word matches and meaning matches both reach the user. Elastic describes it as running full-text search and vector search in one request (Elastic hybrid search page, read October 2026).

Reciprocal rank fusion

Reciprocal rank fusion (RRF) is a method that merges several ranked lists by giving each document the sum of 1 divided by a constant plus its rank in every list it appears in, so documents ranked high in many lists rise to the top without any score comparison (Cormack et al., 2009, at the Special Interest Group on Information Retrieval (SIGIR) conference).

In plain words

Picture two helpers in a library. The first helper matches the exact words on book spines. The second helper understands what you mean, even when you use different words. Ask both, and trust the books both of them mention.

The analogy breaks in one place. The helpers do not give scores on the same scale. In an illustrative case, one says "match strength 2.9" and the other says "closeness 0.81". You cannot add those numbers as they are. The common fix merges the order of the results instead of the numbers. The other fix rescales both sets of scores to one scale first. Order is the one thing both helpers can report in the same way, so this page follows the order-based fix. In one precise sentence: hybrid search retrieves with two methods and fuses their rankings.

Step 1: Score documents by words with BM25

BM25 ranks documents by the words they share with the query. The input is the query text and an index built from the documents. For each query word, BM25 checks how often it appears in a document, how rare it is, and how long the document is. Rare words count more: a code that appears in one document in a million says more than "connection", which appears in many.

For one query word t in one document, the score multiplies the inverse document frequency (IDF) of t by a capped count of t:

Here f is the number of times t appears in the document, L is the document's length in words, avgdl is the average length across all documents, N is the number of documents in total, and n is the number of them containing t. The document's score is the sum over the query words. Here, as with Elasticsearch's default standard analyzer, words are matched after lowercasing with no stemming, so "connections" would not match "connection".

Two settings control the shape. The parameter k1 sets how fast repeated occurrences stop adding score, and b sets how strongly long documents are penalised. Elasticsearch has defaults of 1.2 for k1 and 0.75 for b. With k1 = 1.2, the factor k1 + 1 is 2.2: Lucene 8 and later drops that factor, so Elasticsearch prints scores divided by 2.2 with the same ranking (Elastic similarity settings, read October 2026). BM25 is strong when the query holds the exact words, and blind when it does not.

Step 2: Score documents by meaning with vectors

Vector search ranks documents by how close their embeddings are to the query's embedding. The input is the same query, turned into a list of numbers by an embedding model. The index stores one such list per document, and the search returns the nearest ones. A query such as "why do my requests hang" lands near "slow connections and dropped sessions" with no shared words.

Dense Passage Retriever (DPR), a dense retriever trained on question-passage pairs, beat a strong Lucene BM25 system by 9 to 19 points of top-20 passage accuracy on open-domain question answering, meaning questions on any topic answered from a large text collection (Karpukhin et al., 2020). Top-20 accuracy is the share of questions for which at least one of the 20 returned passages contains the answer. Yet on the BEIR (Benchmarking Information Retrieval) benchmark, which tests retrievers on tasks they were not trained for, the authors report BM25 as a baseline that holds up well (Thakur et al., 2021).

Step 3: Merge the two ranked lists with reciprocal rank fusion

Reciprocal rank fusion merges the two lists by position alone. Each document earns 1 divided by (k plus its rank) from every list that contains it, and the earned amounts are added. The constant k is 60 in the original paper, fixed in a pilot and never changed afterwards (Cormack et al., 2009). Elasticsearch uses the same default of 60 (Elastic reciprocal rank fusion page, read October 2026).

The rule has a clear consequence. A document that is first in only one list earns 1/61, about 0.0164. A document that is second in both lists earns 2/62, about 0.0323, nearly twice as much.¹ Agreement beats a single win: appearing in both lists roughly doubles a document's score. The merge needs no score scale, so it works for any two retrievers.

Step 4: Return one list, and optionally rerank it

The fused list is the output of the system. The input is the two ranked lists, and the output is one list of the top results, cut to the size the product needs. Many systems stop here, because the fusion is cheap and needs no training data.

A common extra stage takes the top few dozen fused results and rescores them with a model that reads the query and each document together, called a reranker; cross-encoder rerankers explains how. Anthropic's retrieval experiments, September 2024, show the layers stack. Failure to find the right passage in the top 20 fell from 5.7% with standard embeddings to 3.7% when each passage was first given added context (contextual embeddings), then to 2.9% when contextual BM25 was added alongside, and to 1.9% after adding reranking. The step that adds BM25 is therefore the move from 3.7% to 2.9%.

1 · Keyword list2 · Vector list3 · Fuse the ranks4 · Top resultsReplay

Step-by-step description

1. BM25 ranks documents by the words they share with the query.
2. The vector index ranks documents by closeness in meaning.
3. Reciprocal rank fusion adds 1 over (60 + rank) per list.
4. One merged list goes to the user or to a reranker.

• BM25 scores documents by the query words they contain.
• Vector search ranks documents by how close their embeddings are to the query embedding.
• Reciprocal rank fusion adds 1 divided by the rank constant plus the rank for each list a document appears in.
• The fused list is returned as the top results.

Worked example

Take six tiny documents and the query "ERR_4021 connection timeout". Document d1 is a fix note that contains the code and explains the timeout. Document d4 is a changelog line that mentions the code. Documents d2 and d3 describe hanging requests and slow connections without the query words. Document d6 explains connection pooling, and d5 is a pasta recipe.

The six texts, with their lengths in words, are: d1 "Fix: error ERR_4021 means the connection timed out; raise the timeout setting" (12), d2 "Why your network requests hang and how to retry them safely" (11), d3 "Troubleshooting slow connections and dropped sessions" (6), d4 "ERR_4021 changelog: added in version 3.2" (7, counting the two parts of the version number separately), d5 "Pasta recipe with tomato and basil" (6) and d6 "Connection pooling settings explained" (4).¹

The average length avgdl is 46 / 6 = 7.667. Of the six, ERR_4021 appears in d1 and d4, "connection" in d1 and d6, and "timeout" only in d1, so IDF is 1.0296 for ERR_4021, 1.0296 for "connection" and 1.5404 for "timeout". There is no stemming, so "connections" in d3 is not a match, and in this toy set ERR_4021 and "connection" share one IDF because each appears in two documents.¹

BM25 was computed exactly with k1 = 1.2 and b = 0.75: d1 scores 2.9237, d6 scores 1.2801 and d4 scores 1.0676; the other three score zero because they share no query word. The vector ranking is illustrative, because no embedding model was run: d2, d3, d1, d6, d4, d5, in that order. Fusing with k = 60 gives d1 0.032266, d6 0.031754, d4 0.031258, d2 0.016393, d3 0.016129 and d5 0.015152.¹

The result teaches two things. The paraphrase d2 was the vector retriever's first choice yet falls to fourth, because BM25 never listed it. That is the cost of rank fusion: a document that only one retriever finds scores low. And the weak keyword match d6 beats d2, since two weak signals add up. Both effects are why teams tune the constant and test on real queries.

Text description of the diagram

• Document d1 has the highest reciprocal rank fusion score, 0.032266.
• Documents d2, d3 and d5 appear in only one list and score about 0.016.
• Appearing in both lists roughly doubles a document's score.

Trade-offs and edge cases

Hybrid search costs two indexes, two queries and a merge, so it adds storage and latency compared with either retriever alone. The table collects the published figures this page relies on, each with its source.

Rank fusion throws away score magnitude. If one retriever is extremely sure and the other is lukewarm, RRF still treats their ranks symmetrically. The alternative is a weighted sum of scores, which keeps magnitude but needs the scores normalised to a common scale first; Elasticsearch's linear retriever supports min-max normalisation for this purpose (Elastic retrievers page, read October 2026).

Other edge cases matter in practice. Keyword search alone may be enough: BEIR found that BM25 holds up well as a baseline. A learned sparse model such as the Sparse Lexical and Expansion model (SPLADE) keeps exact-term matching and adds learned term expansion, with state-of-the-art results reported on BEIR in its v2 paper (Formal et al., 2021). Query rewriting is a third lever: Hypothetical Document Embeddings (HyDE) asks a language model to write a hypothetical answer and embeds that instead of the query (Gao et al., 2022), at the price of a model call before retrieval.

Text description of the diagram

• BM25 finds exact identifiers and vector search can miss them.
• Vector search finds paraphrased questions and BM25 can miss them.
• Hybrid search covers both query kinds but runs two indexes.

The table shows the same trade in a picture: BM25 finds exact identifiers and vector search can miss them. Vector search finds paraphrased questions and BM25 can miss them. Hybrid search covers both query kinds but runs two indexes.

How to answer it in an interview

Give the two failure modes, the merge rule and the cost, in that order. The answer below is an illustrative example of the structure, not a script to repeat.

Say it in this order: the two failures, the merge, the cost, then how you would measure it. It takes about 45 seconds.

Common mistakes

1. Adding raw scores. (wrong) "I add the BM25 score and the cosine similarity (the angle-based closeness of two embeddings)." (right) "The scales differ, so I merge ranks with reciprocal rank fusion, or normalise both scores before a weighted sum."
2. Treating hybrid as always better. (wrong) "Hybrid search always beats either retriever." (right) "It usually helps on mixed queries; I would measure it on our queries, since BM25 alone is a strong baseline on many tasks."
3. Forgetting the cost. (wrong) "It is just two searches." (right) "It needs two indexes, two queries and a merge, and the latency budget has to cover them."
4. Calling BM25 an old idea that embeddings replaced. (wrong) "Embeddings made keyword search obsolete." (right) "Keyword search still wins on exact strings such as codes and names."
5. Skipping evaluation. (wrong) "It looked good on three queries." (right) "I compare recall at a fixed depth (the share of relevant documents found in the top 10, say) for each retriever and for the merged list on a labelled set."

Follow-up questions

What is semantic search?

Semantic search retrieves documents by meaning instead of by shared words, usually by comparing embeddings of the query and the documents. It handles paraphrases well and can miss exact identifiers, which is why hybrid search adds a keyword retriever.

What is hybrid search in RAG?

In retrieval-augmented generation (RAG), hybrid search is the retrieval step that fetches passages with both keyword and vector search before the language model writes its answer. Better retrieval means the model sees the right passage more often, and the AI engineer interview hub places it among the other skills of that role; see agentic AI interview questions for the agent variant.

Hybrid search vs vector search?

Vector search ranks by meaning only; hybrid search adds a keyword ranking and merges the two. Hybrid finds exact strings and paraphrases, at the cost of a second index and a merge step.

How do you combine BM25 and vector scores?

Merge the ranks with reciprocal rank fusion, which needs no common scale, or normalise both scores and take a weighted sum. Rank fusion is the simpler default; the weighted sum keeps score magnitude but needs tuning.

What changes if you set the rank constant to 1 instead of 60?

Top positions dominate. In the worked example, k = 1 gives d1 0.75, d6 0.533, d2 0.5, d4 0.417, d3 0.333 and d5 0.143, so the paraphrase d2 now beats the changelog line d4, which it did not at k = 60.¹

Sources

1. Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods (SIGIR 2009) — Cormack, Clarke and Buttcher, University of Waterloo, 19 Jul 2009
2. Hybrid search — Elastic documentation, 24 Oct 2026
3. Reciprocal rank fusion — Elastic documentation, 24 Oct 2026
4. Similarity settings (BM25 parameters k1 and b) — Elastic documentation, 24 Oct 2026
5. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (arXiv 2104.08663 v4) — arXiv (Thakur et al.), 21 Oct 2021
6. Dense Passage Retrieval for Open-Domain Question Answering (arXiv 2004.04906 v3) — arXiv (Karpukhin et al.), 30 Sep 2020
7. SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval (arXiv 2109.10086) — arXiv (Formal et al.), 21 Sep 2021
8. Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE, arXiv 2212.10496) — arXiv (Gao et al.), 20 Dec 2022
9. Retrievers (linear retriever with min-max normalization) — Elastic documentation, 24 Oct 2026
10. Introducing Contextual Retrieval — Anthropic, 19 Sep 2024