How would you design a system to rank AR Lenses for a user?

In this article

1. Before you start
2. Why interviewers ask this
3. Ranking AR Lenses from first principles
4. The Staff-level answer
5. The rubric
6. Three variants of this question
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This question sits inside the recommendation system design guide, which covers the vocabulary used below.

Why interviewers ask this

The question tests whether a candidate can turn a vague product request into a ranking system with a clear goal, a staged architecture and a plan for learning from its own output. Ranking Lenses looks like a plain recommendation task, so the interviewer can hold the product fixed and probe the engineering: what you optimize, how you keep latency low, and what you do about items with no history.

A weak answer says "train a model on engagement and sort the Lenses by score." A strong answer names the stages, says what each is optimized for, and defines the label. A Staff-level answer adds the failure modes of a system that trains on its own output: it sees feedback only on what it showed, and the metric you can compute quickly is not the one you care about. The question is reported by candidates in design rounds; no company is claimed to ask it. A neighbouring design question is designing a places recommender.

Ranking AR Lenses from first principles

AR Lens

An AR Lens is an augmented reality effect applied to a live camera view, such as a face filter or a world effect, that a user can open, play with, share or save from a large catalog.

Blend score

A blend score is the single number a ranker computes for one user and one Lens by adding the estimated chances of several outcomes, each multiplied by a weight, and subtracting any penalties; the Lenses are then sorted by it.

In plain words

Picture a street market with a million stalls and one friend who knows your taste. The friend removes closed stalls. Then the friend picks a few thousand by instinct. Last, the friend looks closely and puts the best ten in front of you. The analogy breaks in one place. The friend sees the stalls you walk past. A recommender only learns from the stalls it chose to show.

In one precise sentence: for one user and each surviving Lens, the system estimates the chance of outcomes you value. It combines them into a blend score. It shows the top few and logs what happened, so the next model can learn.

Snap's engineering team describes a similar shape for ads: a pipeline of candidate generation, heavy-model scoring, an auction and a feedback loop (Snap Engineering, February 2022). That page covers ads, not Lenses, so the Lens-specific details below are design choices, not reported Snap practice. Covington et al. (2016) split YouTube recommendations into a candidate generation model and a separate ranking model.

1 · Goal2 · Candidates3 · Score4 · Re-rank5 · FeedbackReplay

Step-by-step description

1. Define the label: play, share, save.
2. Filter, then retrieve a few thousand.
3. A multi-task model scores each one.
4. Blend, add variety and new Lenses.
5. Log, evaluate on future data.

• The ranker runs five stages: goal and label, candidates, scoring, re-rank, and logging with evaluation.
• Logging and evaluation feed the next training run.

Step 1: Define the goal and the label

A Lens ranker needs a written goal before it needs a model, and the goal is a label: the outcome you count as success. Raw plays are easy to log but reward Lenses that are opened and dropped. A stronger design counts several outcomes: the user plays the Lens for a meaningful time, shares a creation, or saves the Lens for later. Each becomes its own label.

Input: logged impressions with what the user did next. Output: one label per outcome per impression. Decide the waiting window too: waiting longer gives more complete labels but a staler model, while updating sooner counts late actions as negatives, which are false negatives. Add guardrails you must not hurt, such as complaints or reports. You now have the targets the later stages estimate.

Step 2: Retrieve candidates, including Lenses nobody has seen

Candidate generation is the stage that cuts a large catalog to a small set the heavy model can afford to score, and it is tuned for recall, meaning it rarely drops a Lens the user would love. Eligibility filtering comes first: remove Lenses that fail policy, region or device checks. Lightweight models pick candidates; Snap's ad system keeps hundreds to a few thousand candidates (Snap Engineering, February 2022).

A two-tower model is a pair of networks that turn user features and Lens features into vectors whose dot product is the score. Lens vectors can be computed ahead of time, which keeps retrieval cheap.

The payoff is new Lenses. Selection bias is the distortion that arises when training data covers only the items the system chose to show, so a Lens never shown earns no evidence. Give the Lens tower content features (creator, effect type, tags, early engagement) instead of a bare identifier, add a retrieval source for fresh Lenses, and give them an exploration bonus later.

The sizes are assumptions for the worked numbers, not Snap figures.¹

Step 3: Score each candidate with a multi-task model

The heavy ranker is a model that scores every candidate for this user with richer features than retrieval could afford. Because you care about play, share and save, it estimates each as a separate output, sharing a base network; that is multi-task ranking. Snap's ad models likewise generate several scores per candidate, for example conversion probability.

Input features: the user's history and context, the Lens's content, and their cross terms; for AR add whether the Lens can run on this phone, since a Lens that stutters on the device earns poor outcomes. Output: one estimated probability per outcome per candidate. This stage runs on the whole candidate set, so it is the latency-critical one; retrieval exists to keep its input small. Only the ordering of scores matters for a pure ranking, but where a score is compared with bids or budgets, calibrated scores matter too: a calibrated score is one whose value matches how often the outcome really happens. Snap's auction uses both the ordering and the raw conversion probabilities (Snap Engineering, February 2022).

Step 4: Blend and re-rank for variety and new Lenses

Re-ranking is the final stage that turns per-outcome estimates into the displayed order, and it applies the product rules a model cannot learn alone. The blend score adds each estimated outcome times a weight and subtracts penalties. Weights express value: a share brings new users, so it earns more than a play. Then diversity rules stop ten near-identical face filters, and an exploration bonus gives under-shown Lenses a chance.

The weights are 1 for play, 5 for share and 3 for save, and the bonus is 0.15 for a Lens with few impressions; every number is illustrative. The new Lens rises from 0.36 to 0.51 with the exploration bonus.¹ Ranking by play alone puts Aging filter first and Sparkle trail third; the blend reverses that. The exploration bonus matters because of the selection bias described in Step 2: a Lens never shown can never earn evidence, so the bonus buys it exposure.

Text description of the diagram

• Sparkle trail has the top blend score of 0.71.
• The new Lens rises from 0.36 to 0.51 with the exploration bonus.

Step 5: Log, evaluate on future data and ship by experiment

The feedback stage records what was shown, what the user did and which candidates were not shown, and it decides whether a new model ships. Log the shown list with positions and the scores, since position changes the odds of a play. Snap states that interactions with a shown item generate training data for the models, which is also why the loop can feed on itself.

Evaluate on data after the training period, not a random split. Snap computes metrics on future data, for example the seven days of impressions after the training date range. Then put promising models online for an experiment before a full rollout. A model that wins on past data can still lose in the online experiment, so the experiment decides. Logging and evaluation feed the next training run. You now have the whole funnel.

The Staff-level answer

What makes it Staff-level: it names the label, says what each stage is optimized for, and answers the self-reinforcing loop before the interviewer asks. Say the structure in order: goal, funnel, blend, new Lenses, evaluation.

The rubric

Three variants of this question

How would you rank Lenses for a brand-new user?

Use context and content, not history: device, region, time of day and what is popular among similar new users. Fall back on a popularity-and-freshness list, then shift weight to personal signals as the first plays and shares arrive. This is the user side of cold start.

How would you rank a Lens that was just published?

Retrieve it through content features and a dedicated fresh-Lens source, show it to a small slice of suitable users with an exploration bonus, and let its early outcomes move it up or down. Log the exposure so the next training run has evidence.

How would you also rank sponsored Lenses?

Add a separate stage that combines an advertiser's bid or budget with the estimated outcomes, which makes calibration necessary. Snap's auction uses both the ordering and the raw conversion probabilities, so the scores must be calibrated, meaning each score matches how often the outcome really happens. Keep organic and sponsored value on one scale or the blend becomes arbitrary.

Common mistakes

• "I train one model on plays and sort." (wrong) Plays reward opened-and-dropped Lenses and ignore sharing. (right) "I estimate play, share and save separately and blend them with weights."
• "I score the whole catalog with the big model." (wrong) That is too slow for a live request. (right) "Cheap retrieval cuts the catalog first, and the heavy model scores only the candidates."
• "I use an embedding per Lens identifier." (wrong) New Lenses have none and old ones stop getting traffic. (right) "I add content features and a fresh-Lens retrieval source."
• "I check accuracy on a random split." (wrong) It leaks the future and hides drift. (right) "I evaluate on the days after the training window, then run an online experiment."

Follow-up questions

How do you rank Lenses with several outcomes such as play and share?

Estimate each outcome with its own output of a shared model, then combine them with weights that reflect value. The outputs share a base network, and you treat the weights as a product decision you tune by experiment. A language model can add content understanding to such a ranker, as covered in large language models (LLMs) inside a recommender.

How do you handle new Lenses with no history?

Use content features in retrieval, reserve a few slots for exploration, and log those impressions. The cold start guide covers the options in more depth.

How would you evaluate the ranker before launch?

Train on one period and score the following days, comparing to the current ranker on outcome metrics and guardrails, then run an online experiment. The recommender systems guide lists the usual metrics.

Why not rank by play rate alone?

Play rate rewards curiosity clicks and ignores whether the user enjoyed or shared the Lens, so a high-play, low-share Lens can crowd out better ones. The blend table above shows play-only ordering putting Aging filter first.

What latency budget would you design for?

Ask the interviewer for the product target, then split it across stages, giving the heavy model most of it and keeping retrieval cheap. The number is a product constraint, not a fact to recite.

Sources

1. Machine Learning for Snapchat Ad Ranking — Snap Engineering, 11 Feb 2022
2. Deep Neural Networks for YouTube Recommendations — ACM RecSys 2016 (Covington, Adams, Sargin), 15 Sep 2016