In this article
The vocabulary of recommendation
- Recommender system
- A recommender system is a program that estimates how much each user will like each item from past behaviour and item information, and uses those estimates to choose and order a short list of items to show, such as videos in a feed or products on a page.
- Matrix factorization
- Matrix factorization is a collaborative filtering method that describes every user and every item by a short list of hidden numbers, called factors, learned so that multiplying a user's list by an item's list approximates the rating that user gave or would give.
- Implicit feedback
- Implicit feedback is a kind of evidence of taste that users give without saying so, such as clicks, watch time or purchases, as opposed to explicit feedback such as star ratings; it is plentiful but noisy, because a click does not always mean the user liked the item.
Start here: the learning path
The path begins with this page, which explains the whole system: stages, signals and the problems that follow. It continues with the two-tower model, which is a common way to train retrieval, then the cold start problem, where there is no history for a new user or item. The sections below then cover multi-task ranking, feedback loops and the design prompts interviewers use.
Underneath the whole path sit embeddings, the vectors that place users and items in one space, and the machine learning system design hub, which explains the staged pipeline in general.
Text description of the diagram
- The illustrative catalogue holds ten million items.
- Retrieval keeps five hundred items.
- The ranker keeps fifty and ten are shown.
- What is a two-tower model, and how do you train one?
- What is the cold start problem in recommender systems, and how do you solve it?
- What is multi-task ranking, and how does it work?
- Recommender feedback loops: training data is yesterday's output
- How do you design a places / POI recommender?
- How do you design food-delivery and e-commerce recommendations?
- How do you design a personalized notification ranking system?
- How to recommend the next Confluence page or Jira issue
- How do you design a People You May Know friend and follower recommender?
- How do you design LLMs inside a recommender system?
- What are the models we can start building for a recommender system?
- How would you design a system to rank AR Lenses for a user?
- How do you design Netflix's homepage?
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →What each step of the path teaches
What is a recommendation system, and how does it work from end to end?
A recommendation system works as a funnel: a retrieval stage cuts a catalogue of millions to hundreds, a ranking stage scores those hundreds with a heavier model, and a re-ranking stage adjusts the final order for goals such as variety. Google's Recommendation Systems course names candidate generation, scoring and re-ranking as one common architecture (Google Recommendation Systems course, 2025), and the YouTube paper describes its first two stages, a candidate-generation network that picks hundreds from millions followed by a separate ranking network (Covington et al., 2016).
The reason is cost. With an illustrative catalogue of ten million items, scoring every item with a heavy model for every request is out of reach, so the cheap stage keeps only five hundred and the heavy model runs five hundred times, not ten million. In the illustration the ranker then keeps fifty and ten are shown.1 The machine learning system design hub defines the general candidate funnel.
What is the difference between content-based and collaborative filtering?
Content-based filtering recommends items whose descriptions resemble what the user already liked, while collaborative filtering uses only behaviour. Koren, Bell and Volinsky describe the two strategies: content filtering builds a profile for each user or product from attributes such as genre, and collaborative filtering analyses past behaviour such as transactions or ratings without explicit profiles (Koren et al., 2009). They also note that collaborative filtering is generally more accurate but suffers from the cold start problem for new users and products, where content filtering is superior. The Google course describes both as mapping users and items into a shared embedding space (Google candidate-generation lesson, 2025).
How does matrix factorization learn what users like?
Matrix factorization learns a short vector of hidden factors for each user and each item so that their dot product approximates the rating. Koren et al. report that, in the Netflix Prize competition, such models were superior to classic nearest-neighbour techniques and let a system add implicit feedback, time effects and confidence levels (Koren et al., 2009). To show it working, we generated a synthetic ratings matrix with 200 rows for users and 100 columns for items and three hidden taste factors, kept 15.1% of the cells, trained on 80% of those and tested on the rest.
The held-out error fell from 1.096 (always predicting the global mean) to 0.448; predicting each item's mean rating scored 1.107.1 Item means did no better because our generator gives items no popularity effect; real ratings have one, so Koren et al. add a bias term for each user and each item before the factors (Koren et al., 2009). The model was also given the true number of factors, three, so this is a best case.
| Predictor | Held-out error (root mean squared) |
|---|---|
| Global mean rating | 1.096 |
| Item mean rating | 1.107 |
| Matrix factorization, three factors | 0.448 |
Text description of the diagram
- The global mean has a held-out error of 1.096.
- Predicting each item's mean rating scored 1.107.
- Matrix factorization scores 0.448.
How is the retrieval stage trained?
The retrieval stage is often a two-tower model: one network turns the user and context into a vector, another turns each item into a vector, and the score is their dot product. It learns from logged pairs of a user and an item that user engaged with, so that the engaged item scores higher than the rest of the catalogue; Google's deep-network lesson frames this as a softmax over all items trained with a cross-entropy loss.
Because the item vectors can be computed in advance and searched with a nearest-neighbour index, retrieval stays fast over a huge catalogue. Google's course notes that plain matrix factorization has difficulty using side features such as age or country, and that deep networks can use them (Google deep-network lesson, 2025). Read the full two-tower model explanation.
What is the cold start problem?
The cold start problem is the lack of interaction data for a new user or a new item, which leaves a behaviour-based model with nothing to learn from. Koren et al. name it as the weakness of collaborative filtering where content filtering is superior (Koren et al., 2009). The usual remedies are content features for new items, a short onboarding questionnaire for new users and deliberate exploration. The dedicated page on the cold start problem separates the new-user, new-item and new-system cases and shows the cost of waiting for data.
How do you rank for several goals at once?
A multi-task ranker predicts several outcomes, such as click, watch and like, and combines them into one score. The YouTube watch-next paper says the ranker must balance several, sometimes conflicting, objectives and must cope with implicit selection bias in user feedback; it uses a multi-gate mixture-of-experts layer to share parameters between objectives and a Wide and Deep design to reduce the bias (Zhao et al., 2019).
Why do feedback loops matter?
A feedback loop arises because the items a system shows are the only ones that can collect clicks, so tomorrow's training data is yesterday's model output. Items never shown get no feedback and look worse than they are. Zhao et al. give the example of a click that happens because the current system selected the item, and call the result a feedback loop (Zhao et al., 2019). Remedies are exploration, logging what was shown with its position, and reserving a small share of traffic for items the model is unsure about.
How do you measure whether a recommender works?
Start with saved data: hide part of each user's history, ask the system for a short list, and measure the share of the hidden items it found (recall) and the share of the listed items that were hidden items (precision). Covington et al. report that these batch numbers do not always agree with live behaviour, so the final decision comes from live A/B tests, where one group of users sees the new system and another sees the old one (Covington et al., 2016).
They also rank by expected watch time per impression rather than click-through rate, because ranking by clicks promotes clickbait that people open and abandon. In an interview, name the metric, say how you would check it on saved data, then say how you would confirm it live; the machine learning (ML) system design hub covers A/B tests.
What design prompts do interviewers use?
Interviewers turn the funnel into product prompts: a homepage for a streaming service, notification ranking, people you may know, places nearby, shopping and food delivery, and recommending the next page inside an enterprise tool. Each prompt is the same pipeline with different signals, goals and constraints. The strongest answers state the goal and the metric first, then walk through retrieval, ranking and re-ranking, then name the cold-start and feedback issues.
Questions interviewers ask on recommendation systems
Interviewers ask this topic in three forms. The definition form asks what a recommender is and how collaborative and content-based filtering differ. The trade-off form asks why the system has stages and what is lost by cutting candidates early. The design form gives a product and asks you to build the pipeline. For practice across the whole field read the AI engineer question list and the machine learning system design hub, and prepare a one-minute answer to each form.
Follow-up questions
What is recommendation system?
A recommender system estimates how much each user will like each item and uses the estimates to choose and order a short list; a common design uses retrieval, ranking and re-ranking stages (Google Recommendation Systems course, 2025).
How to design a movie recommendation system?
State the goal, such as watch time or satisfaction, then build the stages: retrieve a few hundred candidates with embeddings, rank them with a model that uses user, movie and context features, and re-rank for variety. Handle new users with content features and a short questionnaire (Koren et al., 2009).
What is recommendation system in ML?
In machine learning (ML) a recommender is a model, or a set of models, trained on past behaviour to score user and item pairs; two common families are matrix factorization and two-tower networks (Google deep-network lesson, 2025).
What is recommendation system in e-commerce?
In e-commerce the same funnel recommends products from purchases, views and basket contents; Koren et al. note that Amazon and Netflix made recommenders a salient part of their sites (Koren et al., 2009).
Why not score every item for every user?
Cost: a heavy model on a catalogue of ten million items for every request is impractical, so a cheap retrieval stage keeps hundreds and the heavy model scores only those; the trade-off is that an item that retrieval drops can never be ranked.
- A recommender is a funnel: retrieval and ranking run costlier models on fewer items, then re-ranking adjusts the short list.
- Collaborative filtering uses behaviour only; content filtering uses item descriptions; each fixes the other's weakness.
- Matrix factorization learns hidden factors; on our synthetic data it cut the held-out error from 1.096 to 0.448.
- New users and items need content features and exploration; shown items alone produce the training data.
- Interview answers start from the goal and metric, then walk the stages.
Sources
- Matrix Factorization Techniques for Recommender Systems — IEEE Computer, vol. 42 no. 8 (Koren, Bell, Volinsky), authors' PDF copy, 1 Aug 2009
- Deep Neural Networks for YouTube Recommendations — Google Research (Covington, Adams, Sargin), RecSys 2016, 15 Sep 2016
- Recommending What Video to Watch Next: A Multitask Ranking System — Google Research (Zhao et al.), RecSys 2019, 16 Sep 2019
- Recommendation systems: deep neural network models — Google for Developers, Recommendation Systems course (updated 2025-08-25), 25 Aug 2025
- Recommendation systems: candidate generation overview — Google for Developers, Recommendation Systems course (updated 2025-08-25), 25 Aug 2025
- Recommendation systems overview: types — Google for Developers, Recommendation Systems course (updated 2025-08-25), 25 Aug 2025
