What is a matrix factorization recommender and how does it work?

In this article

1. Before you start
2. Why interviewers ask this
3. Matrix factorization recommender from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

Background: machine learning algorithms and statistics foundations.

Why interviewers ask this

A candidate who says "factor the ratings matrix" and stops usually hears the next question at once: what happens to the empty cells? Matrix factorization is a common early topic in recommendation design rounds, and candidates report a variant that asks you to compare it with two-tower retrieval for a streaming service.

The question separates people who know the formula from people who know why the formula is shaped that way. A weak answer says "it is singular value decomposition (SVD) on the ratings." A strong answer says the model fits two small matrices only to the observed ratings, adds a regularization penalty, and predicts with a dot product. A Staff-level answer adds the parts that break in practice: biases, implicit data, the cold start problem, a non-convex objective and the point where a deep model takes over.

Matrix factorization recommender from first principles

Matrix factorization recommender

A matrix factorization recommender is a collaborative-filtering model that learns one short vector for every user and one for every item from a sparse table of ratings, so that the dot product of a user vector and an item vector approximates the rating that user gave, or would give, that item.

Bias term

A bias term is a single learned number that captures a systematic effect, such as one user rating everything high or one item being liked by nearly everyone, so the factor vectors only have to model what is specific to a user and item pair.

In plain words

Imagine describing every person by scores on a few taste dials, such as sweet and spicy. Describe every dish by how much sweet and spicy it has. Multiply each dial by the matching amount and add. A high total means a good match. The analogy breaks in one place. In a real model nobody names the dials. The model invents them from data, and Koren, Bell and Volinsky (2009) note that factors can be completely uninterpretable.

In one precise sentence, the model learns a vector p for each user and q for each item so that the dot product of q and p is close to every known rating.

Step 1: Two small matrices replace one large sparse matrix

The raw data is a table with one row per user and one column per item, mostly empty because most users rate few items. The goal is two smaller, dense matrices, a user matrix and an item matrix, whose product approximates the known cells and gives predicted ratings for the empty ones. Koren, Bell and Volinsky describe this as mapping both users and items to a joint latent factor space of dimensionality f.

Their shapes are fixed by the data. Google's recommendation glossary says the user matrix is the number of users by the number of embedding dimensions, and with 10,000 movie titles the item matrix has 10,000 columns. The number of factors f is a choice. You now know the shape: a tall user matrix, a wide item matrix and a small shared width. The next step says how they combine.

Step 2: A dot product plus biases predicts a rating

The prediction for user u and item i is the dot product of the item vector and the user vector. Koren and colleagues write that as

and then extend it with biases: the global average rating $\mu$, an item bias $b_i$ and a user bias $b_u$.

Their example is a movie. The average rating is 3.7 points, Titanic is rated 0.5 above average and Joe rates 0.3 below average, so the estimate before any factors is 3.7 + 0.5 − 0.3 = 3.9 points.¹ The biases absorb the part of a rating that any user or item would show; the dot product then adds only the personal interaction. That is the term candidates drop. A dot product plus biases predicts each rating, and without the biases the vectors waste capacity on averages.

Step 3: Training fits only the ratings we know

The learning objective sums squared error over the known ratings only, plus a penalty on the vector lengths. With $e_{ui}=r_{ui}-q_i^T{p}_u$, the cost is

where the sum runs over pairs with a known rating. This is the basic model without biases; the biased version adds μ, b_u and b_i inside $e_{ui}$ and b_u² and b_i² to the penalty. The user matrix and the item matrix are fit to the known ratings only. The constant $\lambda$ sets the strength of regularization and is usually found by cross-validation, which is holding out some known ratings to test each setting.

Why not plain SVD? Koren and colleagues state that "conventional SVD is undefined when knowledge about the matrix is incomplete." Filling the gaps first, called imputation, is expensive and can distort the data, so recommenders fit the observed cells directly. Two vectors per rating pair can easily memorize a few ratings, and the penalty is what prevents it.

Step 4: Stochastic gradient descent or alternating least squares updates the vectors

Both unknowns, the user vectors and the item vectors, change together, so the objective is not convex. Two methods are standard. Stochastic gradient descent (SGD) loops over the known ratings. For each one it computes the error $e_{ui}$ and nudges both vectors with a learning rate $\gamma$:

Alternating least squares (ALS) is a method that holds the item vectors fixed and solves for the user vectors exactly, then swaps roles and repeats. Holding one side fixed makes each sub-problem quadratic, as Koren and Bell note. Koren and colleagues say SGD is generally easier and faster, but ALS is favorable when you parallelize or when the data is implicit. Stochastic gradient descent or alternating least squares updates the vectors, and each pass lowers the training error.

1 · Two matrices2 · Predict3 · Measure error4 · UpdateReplay

Step-by-step description

1. One sparse matrix becomes a user and an item matrix.
2. A dot product plus biases gives a rating.
3. Only known ratings count in the error.
4. SGD or ALS adjusts the vectors.

• The user matrix and the item matrix are fit to the known ratings only.
• A dot product plus biases predicts each rating.
• Stochastic gradient descent or alternating least squares updates the vectors.

Worked example

A toy matrix shows the mechanics. The ratings below are illustrative: four users, five items, and a dash where nobody rated. Nine of the twenty cells are known, so the model with two factors holds 18 numbers (eight for the four users and ten for the five items) to explain nine ratings.¹

Now take one SGD step on one rating, using the basic model without biases so it matches the Step 4 update rule, with every starting value illustrative. User B rated item 1 as 4. User B's vector is (1.0, 0.5), item 1's vector is (0.6, 0.2), the true rating is 4, the learning rate is 0.1 and the penalty strength is 0.1. The prediction is 0.6 + 0.1 = 0.7, so the error is 3.3.

The new item vector is (0.924, 0.363) and the new user vector is (1.188, 0.561), using the old item vector for the user update.¹ The new prediction is 1.3014 and the error falls to 2.6986: one step moved the guess toward the rating, and training repeats this over every known rating many times.¹ The prediction moves from 0.7 to 1.3014 against a target of 4.

Text description of the diagram

• The prediction moves from 0.7 to 1.3014 against a target of 4.
• One update moves the guess toward the target, and training repeats it over every known rating.

The user and item counts come from the Netflix Prize data of Koren and colleagues, which had more than 100 million ratings on more than 17,000 movies, given by about 500,000 raters. The parameter count is (users + items) × factors; the result and the ratio are our arithmetic.¹ For scale, the same paper reports that Netflix's own system reached a root mean squared error (RMSE) of 0.9514 while the grand prize required 0.8563.

Text description of the diagram

• Netflix own system scored an RMSE of 0.9514.
• The grand prize required an RMSE of 0.8563.

Trade-offs and edge cases

Cold start. The cold start problem is the inability of collaborative filtering to score a new user or a new item, because neither has ratings to learn from. A user with no known ratings never enters the sum over known ratings, so training never updates that vector: it keeps its random starting values and its predictions are noise. Content features, a popularity fallback or deliberate exploration of new items fill the gap.

Implicit feedback. Implicit feedback is data such as clicks, plays or purchases that shows what people did, not what they rated. Hu, Koren and Volinsky (2008) treat it as positive and negative preference with widely varying confidence, because there is little evidence on what users dislike. Koren and colleagues say alternating least squares is favorable when the data is implicit, because the training set cannot then be considered sparse.

Not a global optimum. Because the objective is not convex, a good local solution is what you get, not a proven best one. More factors improved the Netflix error in the paper's figure, and regularization, more than a small factor count, is what controls overfitting.

Text description of the diagram

• Conventional SVD is undefined when knowledge about the matrix is incomplete.
• ALS is favorable when you parallelize or when the data is implicit.

Limits. Matrix factorization uses only user and item identities, so it ignores features and context; deep models such as two-tower networks can use them. Also expect data drift: ratings collected under an old recommender change when a new one ships, a case of distribution shift.

How to answer it in an interview

The spoken structure: name the two matrices, give the prediction with biases, say what you train on and how, then name cold start.

Common mistakes

• "It is just SVD." (wrong) Classical SVD needs a complete matrix. (right) "It is related to SVD but fits only the known ratings with regularization."
• "Empty cells are zero ratings." (wrong) That invents data and drags predictions down. (right) "The loss sums over known ratings; for clicks I use confidence weights instead."
• "The latent factors are genres." (wrong) They can be uninterpretable. (right) "Factors are learned directions; some line up with visible traits, some do not."
• "It solves cold start." (wrong) New users and items have no learned vector. (right) "It does not; I need side features or a fallback."

Follow-up questions

What is the difference between matrix factorization and SVD?

SVD decomposes a complete matrix exactly. A matrix factorization recommender fits two small matrices to observed ratings only, with a regularization penalty, so it works on sparse data.

How does a matrix factorization recommender handle cold start?

It does not on its own, because a new user or item has no learned vector. Use content features, a popularity default or a short onboarding quiz, and learn real vectors as ratings arrive. The collaborative filtering page covers the neighbor-based alternative.

Should missing ratings be treated as zero?

No for explicit ratings: the objective sums only over known ratings. For implicit data, a missing interaction is a weak negative signal handled through confidence weights.

How do two-tower retrieval and matrix factorization differ?

Matrix factorization learns one free vector per user and per item, while a two-tower model computes vectors from features with two networks, so it can score a new item that has features. Matrix factorization is the special case whose towers are just ID embedding lookups, and both end in a dot product. Both let you precompute every item vector and retrieve candidates by nearest-neighbor search on the dot product, so serving cost is similar.

The two-tower model costs more to train and needs feature pipelines for both towers, but it can use context and score items with no interaction history; matrix factorization trains cheaply with alternating least squares or stochastic gradient descent on IDs alone. Choose matrix factorization for a stable catalog with dense history, and two-tower when items churn or rich features exist.

How do you evaluate a recommender: precision at k or NDCG?

Precision at k is the share of the top k recommended items the user liked:

$${\text{precision}}_k=\frac{\text{hits in top}k}{k}$$

Normalized discounted cumulative gain (NDCG) also rewards ranking liked items higher. With ${\mathrm{rel}}_i$ the relevance of the item at rank i, the discounted cumulative gain (DCG) is

$${\text{DCG}}_k=\sum _{i=1}^k\frac{{\mathrm{rel}}_i}{{\log }_2(i+1)}$$

and NDCG divides it by the DCG of the ideal ordering, so a perfect list scores one. Use precision at k when presence in the list matters, and NDCG when order or graded relevance matters; a low rating error does not prove a good ranking.

How do you choose the number of factors and the regularization strength?

Hold out known ratings and pick the settings with the best held-out error, which is cross-validation. This is the bias and variance trade-off: too few factors underfit, too weak a penalty overfits.

Sources

1. Matrix Factorization Techniques for Recommender Systems — IEEE Computer 42(8) (Koren, Bell, Volinsky), DOI 10.1109/MC.2009.263, 1 Aug 2009
2. Matrix Factorization Techniques for Recommender Systems (full-text mirror) — IEEE Computer 42(8) (Koren, Bell, Volinsky), mirror, 1 Aug 2009
3. Collaborative Filtering for Implicit Feedback Datasets — IEEE ICDM 2008 (Hu, Koren, Volinsky), 15 Dec 2008
4. Machine Learning Glossary: Recommendation Systems — Google for Developers, 26 Oct 2026
5. Advances in Collaborative Filtering — Recommender Systems Handbook (Koren, Bell), 1 Jan 2011