In this article
  1. Before you start
  2. Why interviewers ask this
  3. Information retrieval from first principles
  4. Worked example
  5. Trade-offs and edge cases
  6. How to answer it in an interview
  7. Common mistakes
  8. Follow-up questions
  9. Sources

Before you start

Information retrieval is the base of embeddings and vector search. This page teaches the classic keyword pipeline first, because every neural retriever is judged against it. For the neural side, see how transformers work.

Why interviewers ask this

Every search box, document chatbot and retrieval-augmented system starts with a retrieval step, so a wrong mental model here breaks the whole design. Interviewers use the question to see whether a candidate can separate three jobs: finding candidates quickly, ordering them well, and proving the ordering is good.

A weak answer says "it is search" or "it is vector similarity." A strong answer names the pipeline: clean the text, build an index, score matches, return a ranked list, and measure it with metrics that fit a ranking. A Staff-level answer adds the uncomfortable fact that a simple keyword scorer remains a hard baseline to beat on data a model was not trained on, then says what that implies for system design.

Information retrieval from first principles

Information retrieval
Information retrieval is the task of finding material of an unstructured nature, usually text documents, that satisfies an information need from within a large collection, and of putting the best matches first. A web search engine is one example; so is the lookup step of a question-answering system that reads documents.
Inverted index
An inverted index is a data structure that maps each term to the list of documents containing it, called a postings list, so a query reads only the lists of its own terms instead of scanning every document. A search system builds it once from the collection and then updates it as documents change.
BEIR
BEIR (Benchmarking-information retrieval (IR)) is a benchmark that tests retrieval models zero-shot, meaning on datasets they were not trained on, by scoring them across many public datasets from different tasks and domains, so lexical, sparse, dense, late-interaction and re-ranking systems can be compared on equal terms.

In plain words

Think of the index at the back of a textbook. You look up a word and get page numbers. You never read the whole book to find them. The analogy breaks in one place: a book index lists pages in order, while a retrieval system must put the best pages first.

Relevance is how well a document satisfies the information need behind a query. A system cannot see the need, only the words, so it scores words as a stand-in.

In one precise sentence, an information retrieval system turns documents and a query into comparable terms, finds the documents that share them through an index, orders those by a score, and is judged by how many relevant documents it ranks near the top.

Step 1: The text is cleaned into terms

Text is cleaned into terms before indexing. Tokenization is splitting text into small pieces, called tokens, usually words. Case folding turns "Search" and "search" into one term. Stop-word removal drops very common words such as "the." Stemming is chopping word endings so that "retrieving" and "retrieved" share a form; lemmatization does the same with a dictionary. The textbook's early chapters cover this classic chain (review of the book).

Input: raw text. Output: a list of terms per document. The same cleaning must run on the query, or "Running" in the query will never meet "run" in the index. You now know what a term is, which the index needs next.

Step 2: An inverted index maps each term to its documents

An inverted index maps each term to the documents that contain it. Instead of storing each document's words, it stores for each word the list of documents, often with the count of occurrences in each. To answer a query the system fetches the lists of the query's terms and combines them.

The simplest combination is Boolean retrieval. Boolean retrieval is a method that returns the documents matching a logical expression of terms, written with AND, OR and NOT, and ranks nothing: a document either matches or it does not. The reviewed textbook describes it as the dominant approach for many years that is still widely used today (review). Input: query terms. Output: a set of matching documents. A set is not yet an ordering, which is the next step. Searching vectors instead of terms is the job of similarity search and vector indexing.

Step 3: BM25 scores each document and ranks the matches

BM25, the Okapi weighting scheme, scores each matching document and sorts by score. The Stanford textbook introduces it as a probabilistic weighting sensitive to term frequency and document length. For each query term it multiplies two things. The first is a rarity weight, the inverse document frequency (IDF): a term in few documents is a stronger clue than one in many. The second is a term-frequency factor that levels off, so repeats add less each time.

score(d,q)=∑tln(1+N−nt+0.5nt+0.5)·ft(k1+1)ft+k1(1−b+bLdLavg)

Here N is the number of documents, n_t those containing term t, f_t its count in d, and L_d the document length against the average L_avg. The parameter k1 controls how fast the term-frequency factor levels off, and b controls how much the document length adjusts it. Section 11.4.3 of the textbook gives typical values of k1 from 1.2 to 2 and b of 0.75.

The payoff: in the worked example's illustrative collection, one rare-word occurrence scores 7.42 and three common-word occurrences score 3.97.1

Step 4: Metrics grade the ranked list

Metrics such as precision at k and normalized discounted cumulative gain (NDCG) grade the ranked list. Precision is the fraction of retrieved documents that are relevant, tp / (tp + fp). Recall is the fraction of relevant documents that were retrieved, tp / (tp + fn) (Stanford course handout). Both are set-based measures, so for a ranking the textbook takes the top k retrieved documents as the set (chapter 8).

Rank-aware measures follow. For binary relevance the course uses precision at k, Mean Average Precision (MAP) and Mean Reciprocal Rank (MRR); for graded relevance it uses NDCG. NDCG is the discounted cumulative gain (DCG) of your ranking divided by the DCG of the ideal ranking, where each gain is divided by the base-2 logarithm of its rank. Under this form, ranks 1 and 2 are both undiscounted, because the base-2 logarithm of 2 is 1. The ideal ranking is built from every judged document, not only the retrieved ones. The next section runs them.

Information retrieval runs four stages from text to graded rankingFlow of four stages: text is cleaned into terms, an inverted index maps terms to documents, BM25 scores and ranks the matches, and metrics grade the ranked list.CleantexttermsInvertedindexpostingsBM25rankingscoresMetricsgrade thelistDiagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/information-retrieval.html

Four stages turn raw documents into a ranked list you can grade.CC BY 4.0 — source: What is information retrieval, and how does it work?
Step-by-step description
  1. Text becomes terms.
  2. Each term points to its documents.
  3. BM25 ranks the matches.
  4. Metrics grade the ranking.
  • Text is cleaned into terms before indexing.
  • An inverted index maps each term to the documents that contain it.
  • BM25 scores each matching document and sorts by score.
  • Metrics such as precision at k and NDCG grade the ranked list.

Knowing it is half the job. Saying it is the other half.

Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.

Start Free Trial →

Worked example

Take an illustrative collection of 500,000 documents in which "die" appears in 40,000 and "hard" in 300, with k1 of 1.2 and b of 0.75.1 The rarity weights are 2.53 for "die" and 7.42 for "hard", so the rarer word is worth nearly three times as much per occurrence.1

Now score three illustrative documents on the query "die hard". Document A has average length and contains "die" three times but never "hard". Its term-frequency factor is 3 × 2.2 / (3 + 1.2) = 1.57, so it scores 3.97.1 Document B has average length and contains "hard" once and no "die". Its factor is exactly 1.0, so it scores 7.42, almost double document A despite having one matching word against three.1 If document A is twice the average length, the factor drops to 1.29 and the score from 3.97 to 3.27.1

Repeated occurrences of a term give diminishing gains. At average length the term-frequency factor is 1.0 for one occurrence and 1.9643 for ten, below a ceiling of 2.2, which is k1 plus one.1 A document twice the average length scores lower for the same count. The exact IDF form differs between textbooks and libraries; this page uses the one in the formula above. The formula also shows that k1 at 0 reduces the factor to binary matching and b at 0 removes the length adjustment.

BM25 rewards repeated terms less each time they repeatGrouped bars of the BM25 term-frequency factor for 1, 2, 3, 5 and 10 occurrences, with k1 of 1.2 and b of 0.75, for an average-length document and a document twice as long; both climb slowly toward a ceiling of 2.2.Term-frequency factor00.511.522.5123510Occurrences of the termAverage-length documentDocument twice as longDiagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/information-retrieval.html
Extra occurrences of a term add less each time, and a longer document earns less for the same count.CC BY 4.0 — source: What is information retrieval, and how does it work?
Text description of the diagram
  • Repeated occurrences of a term give diminishing gains.
  • A document twice the average length scores lower for the same count.

Metrics next, on an illustrative ranking of five documents where positions 1, 3 and 5 are relevant and the collection holds four relevant documents in all.1

On this illustrative ranking precision falls and rises with the cutoff while recall at five is 0.75, and graded NDCG is 0.71.
QuantityValue (computed, illustrative input)
Precision at 30.67 (2 of 3)
Precision at 40.5 (2 of 4)
Precision at 50.6 (3 of 5)
Recall at 50.75 (3 of 4)
Average precision0.57
Reciprocal rank of the first hit1.0
DCG of grades 2, 0, 3, 0, 14.32
DCG of the ideal order 3, 2, 1, 1, 06.13
NDCG0.71

Average precision averages the precision at each relevant hit, here the precisions at the three hits (1, two thirds and three fifths) added and divided by the 4 relevant documents, which is 0.57.1 The course handout averages the same hit precisions over the three hits and gets 0.76; this page divides by all 4 relevant documents. For graded relevance, the grades 2, 0, 3, 0, 1 give a DCG of 4.32 against 6.13 for the ideal order, so NDCG is 0.71.1

Trade-offs and edge cases

BM25 is a strong zero-shot baseline, while most neural retrievers are stronger in domain. The BEIR authors evaluated 10 systems on 18 datasets: they call BM25 a strong baseline, re-ranking and late-interaction models achieve the best zero-shot results on average at high computational cost, and dense and sparse models are cheaper but often underperform other approaches. In the paper's Table 2, on in-domain MS MARCO (Microsoft Machine Reading Comprehension), a web-search dataset, BM25 scores 0.228 NDCG@10, against 0.296 to 0.413 for most neural models, though Dense Passage Retrieval (DPR) scores 0.177 (BEIR full text, Table 2).

Results vary by dataset: on Text REtrieval Conference (TREC)-COVID (Text REtrieval Conference, coronavirus disease 2019 (COVID) collection), BM25 scores 0.656 against 0.332 for DPR (BEIR full text, Table 2).

That evidence dates from 2021, and the field has moved since. Learned sparse models such as SPLADE (SParse Lexical AnD Expansion model) predict term weights with a neural network but keep the inverted index (Formal et al., 2021). MTEB (Massive Text Embedding Benchmark) scores embedding models across many tasks, and its retrieval task reuses BEIR datasets (Muennighoff et al., 2022). Check a current leaderboard (as of October 2026) rather than rely on the 2021 numbers.

The figure below shows that Boolean retrieval returns a set, BM25 and dense retrieval rank candidates, and a cross-encoder reranks a short list.

Retrieval methods differ in what they match and how they rankTable comparing Boolean retrieval, BM25, dense retrieval and a cross-encoder by what each matches on, how each ranks, and the stage where each is typically used.ApproachMatchesonRanks byTypicalstageBooleanExactterms, setlogicMatch or nomatchFilters,exactlookupsBM25SharedtermsWeightedterm scoreFirst-stageretrievalDenseVectorclosenessSimilarityscoreFirst-stageretrievalCross-encoderQuery andtext readtogetherLearnedrelevancescoreReranking ashort listDiagram: Interview Clozo, CC BY 4.0 ·interview.clozo.ai/blog/information-retrieval.html
Boolean, BM25 and dense methods retrieve; a cross-encoder usually reranks a short list.CC BY 4.0 — source: What is information retrieval, and how does it work?
Text description of the diagram
  • Boolean retrieval matches exact terms and returns a set.
  • A cross-encoder reads the query and the text together to rerank a short list.

Boolean retrieval matches exact terms and returns a set. A cross-encoder reads the query and the text together to rerank a short list; see the cross-encoder reranker page and the difference between bi-encoders and cross-encoders. Dense retrievers such as the two-tower model match by vector closeness, which helps when query and document use different words for one idea (semantic similarity).

Edge cases: NDCG has variants, and one divides by the base-2 logarithm of the rank plus one, which discounts every rank and gives different values, so state which form you use; a stemmer can merge unrelated words; a rare typo gets a huge rarity weight; very short documents make length normalization aggressive; and set-based precision hides order, which is why ranked lists need rank-aware metrics. Many production systems combine both families, as described in hybrid search.

How to answer it in an interview

Interviewer asks

What is information retrieval, and how would you build and evaluate a basic system?

A strong answer

Information retrieval is finding the documents in a large collection that satisfy an information need, ranked by relevance. I clean the text into terms, build an inverted index that maps each term to its documents, score matches with BM25, which weights rare terms and levels off on repeated ones, and return the top k. I measure it with precision and recall at k, and NDCG when relevance is graded. Most neural retrievers win in domain, but BM25 is a strong zero-shot baseline, so I keep it and often add a reranker.

The spoken structure: define it, walk the pipeline in order, name the metrics, then give the baseline trade-off.

Common mistakes

  • "Information retrieval is just web search." (wrong) The definition covers any large collection of unstructured material. (right) "Web search is one application; document stores and question-answering lookups are others."
  • "Dense retrieval always beats BM25." (wrong) On BEIR (2021), BM25 is a strong zero-shot baseline, and results still vary by dataset. (right) "Most dense models win in domain; I still test against BM25 on my own data."
  • "Boolean search is obsolete." (wrong) The textbook review calls it still widely used today. (right) "Boolean filters are still useful for exact constraints."
  • "Ten mentions of a word means ten times the relevance." (wrong) BM25 levels off. (right) "Term frequency saturates, and long documents are normalized."
  • "Precision and recall are enough for a ranking." (wrong) They are set-based. (right) "I add precision at k, MAP, MRR or NDCG."

Follow-up questions

How do you evaluate the quality of embeddings for information retrieval?

Build a set of queries with documents labeled relevant, embed both, retrieve the top k and compute recall at k, MRR or NDCG. Compare against BM25 on the same queries, and test on data outside your training domain, because the BEIR authors report that dense and sparse models "often underperform other approaches", "highlighting the considerable room for improvement in their generalization capabilities" (BEIR). Pure similarity checks are covered under semantic similarity.

What is the difference between precision and recall in information retrieval?

Precision asks how much of what you returned is relevant; recall asks how much of what is relevant you returned. In the worked example, precision at 5 is 0.6 and recall at 5 is 0.75 for the same list.

Why does BM25 still matter when you have dense retrieval?

It needs no training, handles exact terms and rare names well, and BEIR treats it as a strong baseline for generalization. It is also the other half of hybrid search.

Run both retrievers, merge their candidate lists with a fusion rule, then rerank. The hybrid search page shows the merge.

What happens if you set b to 0 or raise k1?

Setting b to 0 removes the length adjustment, so long documents are no longer penalized for their length. Raising k1 delays saturation and lifts the ceiling of the term-frequency factor, which is k1 plus one: 2.2 at k1 of 1.2, 3 at k1 of 2.

How do you improve a ranking after the first stage?

Rerank the short list with a cross-encoder, which reads query and document together; the BEIR authors report it among the best zero-shot performers at high computational cost.

Key takeaways
  • Information retrieval ranks documents from a large collection against an information need.
  • The classic pipeline is clean text, inverted index, BM25 scoring, then evaluation.
  • BM25 weights rare terms and levels off on repeated ones, so a rare word can outscore a common one repeated.
  • Precision and recall are set-based; ranked lists need cutoff or rank-aware metrics such as NDCG.
  • BM25 stays a strong zero-shot baseline, and a reranker usually follows the first stage.

Sources

  1. Introduction to Information Retrieval (online edition), Manning, Raghavan and Schütze — Stanford NLP / Cambridge University Press (accessed 2026-10-26), 1 Jan 2009
  2. Okapi BM25: a non-binary model, Introduction to Information Retrieval, section 11.4.3 — Stanford NLP / Cambridge University Press (accessed 2026-10-26), 1 Jan 2009
  3. Evaluation in information retrieval, Introduction to Information Retrieval, chapter 8 — Stanford NLP / Cambridge University Press (accessed 2026-10-26), 1 Jan 2009
  4. CS276 evaluation lecture handout — Stanford University (accessed 2026-10-26), 1 Jan 2014
  5. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models (arXiv 2104.08663) — arXiv (Thakur et al., accessed 2026-10-26), 17 Apr 2021
  6. Review of Introduction to Information Retrieval, Information Retrieval Journal — Springer (accessed 2026-10-26), 1 Jan 2009
  7. BEIR full text (arXiv 2104.08663v4) — arXiv (Thakur et al., accessed 2026-10-26), 17 Apr 2021
  8. MTEB: Massive Text Embedding Benchmark (arXiv 2210.07316) — arXiv (Muennighoff et al.), 13 Oct 2022
  9. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking (arXiv 2107.05720) — arXiv (Formal et al.), 12 Jul 2021