In this article
Before you start
This page is one step of the machine learning (ML) system design interview guide.
Why interviewers ask this
A candidate who draws one large model over every item has already broken the latency budget, so the funnel is usually the first box on the whiteboard in a recommendation or search design round. Candidates report prompts of this kind: a home feed over billions of videos with a budget of about one second, a question about which model architecture to use at each stage, and a question about how to choose the number of candidates per request.
A weak answer names three stages and stops. A strong answer says why each stage exists, which metric each one is tuned for, and what the earlier stage can never repair. A Staff-level answer adds the budget arithmetic and says which meaning of "re-ranking" it is using.
Retrieval, ranking, re-ranking from first principles
- Ranking model
- A ranking model is the second-stage model that scores each candidate using a rich set of features about the user, the item and the context, so that the candidates can be put in order and only the best few kept.
- Re-ranking
- Re-ranking is the last adjustment of an ordered list before it is shown: filters remove items, scores are transformed, and criteria such as diversity, freshness and fairness are applied. In search, the same word often names a second-stage scoring model instead.
In plain words
Picture a recruiter with a hundred thousand curricula vitae (CV). First they skim for keywords and keep a few hundred. A panel then interviews those few hundred in depth and picks a shortlist. Finally the hiring manager adjusts the shortlist so the team is not ten copies of one profile.
The analogy breaks here: a rejected CV is gone for good, but the funnel runs again for every request in a fraction of a second, so a dropped item is lost only for this request. Within one request, though, no later stage can bring it back. That one-way door shapes every design choice below.
In one precise sentence, the funnel runs a cheap model over many items, a more precise model over the survivors, and rules over the final list.
Step 1: Retrieval cuts billions of items to a few hundred or thousand
Retrieval is the stage that turns a very large corpus into a small candidate set quickly. Google's recommendation overview describes a first stage that starts from a potentially huge corpus and generates a much smaller subset, and says the YouTube candidate generator reduces billions of videos to hundreds or thousands (accessed October 2026).
Input: a query or a user. Output: a candidate list. Retrieval usually compares embeddings with a similarity measure, and the course notes that the dot product is sensitive to the norm of the embedding while the cosine is not. Several generators with different sources can feed one pool.
Step 2: Approximate search keeps retrieval fast
To stay fast over billions of items, the item embeddings are computed ahead of time and stored in an approximate nearest-neighbour (ANN) index, a structure that returns the vectors close to the query vector without comparing the query with every item. In recommendation, the embedding model with one network for the user or query and one for the item is called a two-tower model, the recommendation name for a bi-encoder. Covington et al. (2016) describe serving candidates with a nearest-neighbour search in dot-product space.
Recall is the share of the truly relevant items that the stage lets through; retrieval is tuned for it. You now know where items enter the funnel; next comes the stage that orders them.
Step 3: Ranking scores each candidate with a richer model
Ranking is the stage in which a more precise model scores the small candidate set. Google's recommendation overview explains that, because the set is small, the system can afford a more precise model, and that the shown list is on the order of ten items (accessed October 2026).
Input: the candidate pool, merged from every source. Output: one score per item and an order. Covington et al. (2016) write that the ranking network assigns each video a score using a rich set of features describing the video and the user. The same paper says the candidate network provides only broad personalisation, which is why the funnel does not let the candidate score decide the final order.
Precision is the share of the shown items that are truly relevant; ranking is tuned for it at the top of the list. Next: a third stage changes the list again, and its name is used in two ways.
Step 4: Re-ranking means two different things
Re-ranking names two jobs, and an interviewer may mean either. In recommendation, Google's re-ranking page describes it as the final stage that considers extra criteria or constraints, by filters that remove candidates or by manually transforming the ranker's score (accessed October 2026). It also lists diversity, freshness and fairness as goals.
In search, the Sentence Transformers documentation uses "re-ranker" for a second-stage model that reads the query and a document together and outputs one relevance score. That model is called a cross-encoder, and the first-stage embedding model is called a bi-encoder.
The mapping: the search re-ranker is the ranking stage of the recommendation funnel. Say "I will call the constraint stage re-ranking and the precise scorer the ranker" and the confusion disappears.
Step 5: Each stage is judged by a different yardstick
The funnel works because each stage has its own job and its own metric. Covington et al. (2016) describe the candidates as meant to be generally relevant to the user with high precision, and ranking as telling their relative importance apart with high recall. They use the words the other way round; the common convention, used on this page, is recall at the candidate cut and precision at the top of the list.
A practical reading: measure retrieval by recall at the candidate size, ranking by quality at the top of the list, and re-ranking by the constraint it enforces, such as diversity. The cap matters most. Whatever retrieval drops, no later stage can bring back, so end-to-end recall can never exceed retrieval recall. Evaluate each stage on a held-out set before you tune the next one.
Step-by-step description
- A fast stage cuts the corpus to candidates.
- A richer model scores the candidates.
- Filters and diversity adjust the order.
- Each stage is judged by its own metric.
- Retrieval cuts the corpus to a small candidate set.
- Ranking scores the candidates with a richer model.
- Re-ranking applies filters and diversity before the list is shown.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
The corpus size below comes from a prompt that candidates report (billions of videos); every cost and recall figure is assumed for illustration and is not a measurement of any real system.1 Take 10,000,000,000 items, a ranking model that costs 50 microseconds per item, and a cross-encoder that costs 5 milliseconds per query-item pair (all assumed).
Scoring the whole corpus with the ranking model costs 500,000 seconds of compute, about 5.79 days, for one request.1 Scoring 1,000 candidates costs 50 milliseconds. The cross-encoder over the whole corpus would take about 578.7 days, against half a second for 100 pairs. Retrieval is what makes the precise stages affordable: it shrinks the work by a factor of 10,000,000 at 1,000 candidates.
| Candidates passed on (assumed) | Retrieval recall (assumed) | Ranker compute per request | Extra recall per added 100 candidates |
|---|---|---|---|
| 100 | 0.62 | 5 milliseconds | not applicable |
| 300 | 0.78 | 15 milliseconds | 0.08 |
| 1,000 | 0.90 | 50 milliseconds | 0.0171 |
| 3,000 | 0.95 | 150 milliseconds | 0.0025 |
If the ranker keeps 70% of the relevant items that reach it in its top ten (assumed), end-to-end recall is 0.90 × 0.70 = 0.63.1 The table also shows why a fixed candidate count is a compromise: the marginal recall per candidate falls by a factor of about 30 between the first and last row, and a system can spend more candidates on requests where recall is hard.
Text description of the diagram
- Assumed sizes are 10,000,000,000 items, 1,000 candidates and 10 shown.
- Retrieval shrinks the work by a factor of 10,000,000 at 1,000 candidates.
Trade-offs and edge cases
The funnel trades recall for speed. Every extra candidate raises the chance the right item survives and raises the ranker's cost, and the worked table shows returns shrinking.
Text description of the diagram
- Recommendation courses use re-ranking for filters, freshness, diversity and fairness.
- In search, a re-ranker is a second-stage model that reads the query and a document together and outputs one relevance score.
Edge cases that change the answer:
- Similar candidates. Google's re-ranking page warns that always recommending the items closest to the query embedding makes the candidates very similar. Its fixes are several candidate generators, several rankers with different objectives, and re-ranking by metadata such as genre.
- Popularity bias. The dot product favours items with large embedding norms, which tend to be frequent items (Google, accessed October 2026); cosine ignores the norm.
- Accuracy gains. Nogueira and Cho (2019) report that a BERT (Bidirectional Encoder Representations from Transformers) re-ranker beat the previous best on the MS MARCO passage ranking task by 27% relative in mean reciprocal rank at 10 (mean reciprocal rank (MRR)@10).
- Feedback loops. When users see only what the system serves, training data reflects past choices. Chaney et al. (2018) study this in simulation and report that such feedback loops make user behaviour more homogeneous; mention it and propose exploration traffic.
For a full application of the funnel, see the search ranking system design. The embedding side of retrieval is covered in designing a multimodal embedding model and its service.
How to answer it in an interview
Walk me through retrieval, ranking and re-ranking for a feed.
I use a funnel because scoring every item with a precise model is too slow. Retrieval uses a two-tower embedding model whose item vectors sit in an approximate nearest-neighbour index, so I never compare against every item, plus a few other sources, to cut billions of items to hundreds or thousands, and I tune it for recall. A ranking model with rich user, item and context features scores that pool and keeps about ten, tuned for precision at the top. A final re-ranking step applies filters, freshness and diversity. The trade-off is that retrieval recall caps everything after it, so I would measure recall at the candidate size first.
The spoken structure: why a funnel, then each stage with its metric, then the cap on recall.
Common mistakes
- "Retrieval must be the most accurate stage." (wrong) It only needs speed and high recall. (right) "Retrieval is tuned for recall; precision is the ranker's job."
- "Nearest to the query embedding means best." (wrong) Nearest items are often near-duplicates. (right) "I add diversity in re-ranking and use several candidate sources."
- "Cosine and dot product behave the same." (wrong) The dot product depends on the norm. (right) "The dot product favours popular, large-norm items, so I choose the measure on purpose."
- "Re-ranking is the cross-encoder." (wrong) Only in search wording. (right) "In a recommendation funnel re-ranking applies constraints; the cross-encoder is a ranking stage."
Follow-up questions
What is the difference between candidate generation and ranking?
Candidate generation cheaply cuts a huge corpus to a small pool and is judged on recall. Ranking spends more compute per item, using richer features, to order that pool and is judged on the top of the list.
What model architectures would you choose for each stage?
For retrieval, an embedding model that scores items with a similarity measure, such as a bi-encoder (Sentence Transformers) or a network over user history as in Covington et al.. For ranking, a model over rich features, or a cross-encoder in search. Re-ranking is mostly rules.
Is learning-to-rank the only type of ranking to consider?
No: learning-to-rank has pointwise, pairwise and listwise approaches (Cao et al., 2007). Pointwise methods score each item alone, such as a predicted click probability; pairwise methods learn which of two items ranks higher, as RankNet and LambdaRank do (Burges, 2010); listwise methods optimise a list metric such as normalised discounted cumulative gain (NDCG), and LambdaMART, the boosted-tree version of LambdaRank (Burges, 2010), reaches that goal through its lambda gradients. Candidates report a prompt that asks for the choice between pairwise and pointwise loss, and strategies that are not learned rankers include rules and score transforms (Google's re-ranking page), blending several objectives, and exploration traffic.
Do you want to think about any other ranking strategies here?
Yes: several rankers trained on different objectives, and re-ranking by metadata for diversity, are both listed in Google's re-ranking page.
How do you choose how many candidates to pass to the ranker?
Fix the latency budget, then pick the count where extra recall per added candidate falls below its cost, as in the worked table. Candidates report a ByteDance prompt that asks for a per-request count instead of a fixed one. A nearby design question is typeahead autocomplete.
Why does a funnel need re-ranking after ranking?
Ranking optimises one score, so the top of the list can be repetitive, stale or disallowed. A final stage fixes that with filters and adjustments. The computer vision system design questions are a neighbouring topic where the same cost trade-off between cheap and costly models can come up.
- The funnel runs cheap models on many items and costly models on few.
- Retrieval is tuned for recall and speed; ranking for precision at the top.
- Re-ranking means final constraints in recommendation and a cross-encoder scorer in search.
- Retrieval recall caps end-to-end recall, because dropped items never return.
- Choose the candidate count from the latency budget and the falling marginal recall.
Sources
- Recommendation systems overview: types — Google for Developers (accessed 2026-10-26), 26 Oct 2026
- Scoring (recommendation systems) — Google for Developers (accessed 2026-10-26), 26 Oct 2026
- Re-ranking (recommendation systems) — Google for Developers (accessed 2026-10-26), 26 Oct 2026
- Candidate generation overview — Google for Developers (accessed 2026-10-26), 26 Oct 2026
- Deep Neural Networks for YouTube Recommendations — Google Research (Covington, Adams, Sargin; RecSys 2016), 15 Sep 2016
- Retrieve & Re-Rank — Sentence Transformers documentation (accessed 2026-10-26), 26 Oct 2026
- Passage Re-ranking with BERT (arXiv 1901.04085) — arXiv (Nogueira and Cho), 13 Jan 2019
- From RankNet to LambdaRank to LambdaMART: An Overview (MSR-TR-2010-82) — Microsoft Research (Burges), 23 Jun 2010
- Learning to Rank: From Pairwise Approach to Listwise Approach (ICML 2007) — Microsoft Research (Cao, Qin, Liu, Tsai, Li), 20 Jun 2007
- How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility (arXiv 1710.11214) — arXiv (Chaney, Stewart, Engelhardt), 30 Oct 2017
