From business goal to ML task: how do you frame the problem?

In this article

1. Before you start
2. Why interviewers ask this
3. From business goal to ML task, from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one part of the ML system design interview guide, which shows where problem framing sits in the whole round.

Why interviewers ask this

A candidate who starts with a model name has usually skipped the part of the round that decides the answer. Interviewers use an under-specified prompt to see whether you ask what the product needs before you design anything. Candidates report this pattern in several prompts: a recommender for a streaming service, a taxi-trip dataset to turn into something useful for drivers within a week, and a question about which customer questions to ask before designing at all.

A weak answer names an algorithm and an offline score. A strong answer states the outcome, says what the label really measures, picks an output type, and defines success in business terms with a baseline. A Staff-level answer adds timing and cost: when the prediction must arrive, what the decision does with it, and what the simplest non-ML version would achieve.

From business goal to ML task, from first principles

ML task

An ML task is a precise statement of what a model takes as input, what it outputs, and which label it learns from, written so that the output feeds one concrete product decision that serves the business goal.

Proxy label

A proxy label is a column used as the training target because the quantity you really want is not recorded, such as clicks standing in for usefulness. A proxy does not measure the goal directly, so it carries a known failure mode.

Success metric

A success metric is a business or product number, such as the share of support cases needing a human, that defines whether the feature worked. It is separate from model evaluation metrics such as precision or recall.

In plain words

Suppose your goal is to get fit. A fitness app cannot measure fitness directly. So it predicts tomorrow's step count, and the step counter is the label. That works until someone shakes the counter. The analogy breaks there. A step counter is gamed on purpose. A model's proxy drifts away from the goal without anyone gaming it. The recorded number simply rewards something slightly different from what you want.

In one precise sentence, framing is choosing an ideal outcome, a label that measures it, an output type, and a success metric that you can check without trusting the model's own score.

Step 1: State the ideal outcome and check that ML is needed

The ideal outcome is the exact task the product should perform, written without mentioning any model. Google's framing-an-ML-problem page asks for the ideal outcome independent of the model, and the summary adds checks that the use case is clear and the data is understood.

Then ask whether ML is needed. Zinkevich's Rules of Machine Learning open with their first rule: do not be afraid to launch a product without machine learning. A rule, a lookup table or a count often comes first.

The ideal outcome is stated before any model is chosen. Input: a business goal. Output: one sentence naming the outcome and the simple non-ML version. You now have a target the later steps must serve.

Step 2: Choose the label and know what the proxy rewards

The label is the column the model learns to produce. When the ideal outcome is not recorded, a proxy label stands in. Google's framing-an-ML-problem page warns to be cautious with proxy labels because they do not directly measure what you want to predict.

Google's table lists proxies and their issues for "recommend useful videos". Most users never click like. A popular label is not personalized. Predicting that a user watches a lot of the video favors long videos over short ones. A rewatch count favors rewatchable genres. (Illustrative: a click-on-play label would reward clickbait.)

Input: the ideal outcome and the columns you have. Output: one label and one sentence on what it rewards and how it fails. Label quality matters too: the problem-understanding page notes that if more than a small percentage of labels are wrong, predictions turn poor.

Text description of the diagram

• Most users never click like.
• A popular label is not personalized.
• Predicting that a user watches a lot of the video favors long videos over short ones.
• A rewatch count favors rewatchable genres.

Step 3: Pick the output type and the decision it feeds

The model output is a number, a class or generated content. For recommenders, ranking is the usual framing: the model scores each candidate pointwise, for example the probability of a play, and the product then sorts the candidates by that score. A regression model predicts a numerical value, a classifier picks a class, and a generative model produces content from the user's intent, such as a summary.

Then say what the product does with the output. Google's framing-an-ML-problem page notes that an app might turn a numeric prediction into a decision, for example doing one thing if the value falls in one range, and that sometimes the prediction is itself the app's output. A threshold is part of the task, not a detail added later.

Input: the label. Output: an output type, a decision rule, and a deadline for when the prediction must exist. You now know what the model hands to the product, which sets up the final step: judging whether that handoff worked.

Step 4: Define success metrics and a baseline

A success metric reflects the business outcome and is distinct from model evaluation metrics, according to Google's framing-an-ML-problem page. Write what counts as success and what counts as failure. In the same page, Google's examples include users opening a weather feature 50 percent more often, and average time on a video site rising 20 percent.

A good metric is quantifiable. Google's project guidance gives one: support cases needing human involvement falling from 72% to 50%. Zinkevich's Rule 13 advises a simple, observable and attributable first objective, and Rule 12 says not to overthink which objective you optimize.

Input: the decision from step 3. Output: a business metric, a pass line, and a baseline to beat.

Which offline metric, and how it links to the business metric

An offline metric is a score computed on held-out data before launch, and you choose it to match the output type. For a ranked list, precision at k, the share of the top k items that are relevant, or normalized discounted cumulative gain (NDCG), which rewards putting relevant items higher, fits. For a binary score, the receiver operating characteristic (ROC) curve plots the true positive rate against the false positive rate as the threshold moves, and the area under the ROC curve (AUC) summarizes how well the score separates the two classes.

None of these is the success metric. Validate the link by a live A/B comparison, which randomly splits users between the new and the old version over the same period and compares the business metric. Add guardrail metrics, numbers that must not get worse, such as complaints or latency. If a better offline score does not move the business metric, the framing is wrong, not the model.

1 · Outcome2 · Label3 · Output4 · MetricReplay

Step-by-step description

1. State the ideal outcome without a model.
2. Choose the label and name its failure mode.
3. Pick the output type and the decision.
4. Define success apart from model scores.

• The ideal outcome is stated before any model.
• The label measures the outcome, often by proxy.
• Success metrics are business numbers separate from model scores.

Worked example

One prompt reported by candidates asks you to take a dataset of taxi trips (fare, locations, times, distance) and propose something that helps drivers and could ship within a week. No public source gives data for it, so the numbers below come from a synthetic dataset that we generated; they are illustrative and show the method, not real taxi demand.¹

Frame it. The ideal outcome, which you state as an assumption and confirm with the interviewer, is that drivers spend less time without a passenger. The data already records pickups, so the label is trips started per zone per hour, with no proxy needed. The output is a number, a regression, and the decision is to show a driver the busiest zone for the coming hour. The success metric is idle time per driver, measured in a live comparison, not the model's error.

The synthetic data covers 4 zones over 28 days and holds 35,575 trips. We train on the first 21 days and test on the last 7, which is 672 zone-hours.¹ Mean absolute error (MAE) is the average size of the miss, here in trips per zone-hour. A baseline is a simple method a model must beat, and three are scored below.

Two lessons follow from this illustrative run.¹ The hour-of-day table is the one-week solution: three weeks of counts and a lookup, with no training pipeline. And the "busiest zone" decision looks solved by a constant, so the model's error is not the success metric. Only a comparison of driver idle time, with and without the suggestions, shows value.

Text description of the diagram

• The flat average has a mean absolute error of 6.078 trips per zone-hour.
• The hour-of-day average has 2.799 and the same hour last week has 3.682.

Trade-offs and edge cases

Framing fails in predictable ways, and each has a fix you can say aloud.

• Model success is not business success. Google's success page warns that a model with impressive metrics can still fail to improve the business metric. Check the metric you report is the one the business owns.
• Timing. A correct prediction that arrives after the decision has no value, so state the deadline in step 3.
• Feedback loops. If the model's output changes the data it later trains on, it can distort itself: sending every driver to the busiest zone oversupplies that zone and starves the others.
• Representative data. Google's problem-understanding page asks that datasets be as representative of the real world as possible, so ask who is missing from the training data.
• Simple first. Launching with a heuristic is acceptable; the model then has a baseline to beat. For a worked framing of a related task, estimated time of arrival (ETA) prediction, see designing ETA prediction and dynamic surge pricing.

How to answer it in an interview

The spoken structure: outcome, label and its failure mode, output and decision with its deadline, then the business metric and baseline.

Common mistakes

• "Our area under the ROC curve (AUC) is high, so the project worked." (wrong) Model scores and business results are different things. (right) "AUC tells me the model ranks well; the business metric tells me whether it helped."
• "I'll predict clicks, since clicks are what we have." (wrong) The proxy may reward clickbait. (right) "Clicks are a proxy; I'd check them against a closer signal."
• "I'll pick the best algorithm first." (wrong) Framing defines outcome, output and metrics, not the algorithm. (right) "I'll define the task and baseline, then choose a model."
• "I'll find the perfect objective before starting." (wrong) Zinkevich advises not to overthink it. (right) "I'll start with a simple, observable, attributable objective and refine it."

Follow-up questions

What functional and non-functional requirements would you clarify first?

Functional requirements say what the system does: what it recommends, to whom, and in which surface. Non-functional ones set the limits: how fast the answer must arrive, how fresh the data must be, the cost per request and the privacy rules. Ask these before choosing a model, because they fix the deadline in step 3.

What signals would you feed into the recommendations?

Start from the label you chose and list signals that exist at the moment of prediction: what the user did before, what the item is, and the context. Drop any signal that is only known after the outcome, because it leaks the answer. A search-focused version of this question is covered in designing a search ranking system.

Would you use hover-over behavior as a signal?

Yes, as a weak implicit signal, not as the label. Hovering shows interest but also accidental movement, so check it against a stronger outcome such as a play or a rating before trusting it.

What are the limits of click-through rate as the only online metric?

Click-through rate (CTR) is a proxy: it can reward clickbait rather than usefulness. Pair it with a satisfaction measure and a guardrail metric that must not get worse.

How would you ship a taxi-trip solution for drivers within a week?

Frame the outcome, then ship the simplest version: hour-of-day averages per zone, as in the worked example, shown to drivers. Run a live comparison of idle time while you build a model that has to beat it.

What if the prediction arrives too late to act on?

Then the task is wrong, however accurate the model. Move the prediction earlier, change the decision to one that still works at that delay, or reconsider ML. Fraud detection and harmful-content detection both turn on this deadline.

Sources

1. Problem Framing: Summary — Google for Developers (Machine Learning), 25 Aug 2025
2. Framing an ML problem — Google for Developers (Machine Learning), 25 Aug 2025
3. Success (Managing ML projects) — Google for Developers (Machine Learning), 25 Aug 2025
4. Understand the problem (Problem Framing) — Google for Developers (Machine Learning), 25 Aug 2025
5. Rules of Machine Learning: Best Practices for ML Engineering — Google for Developers (Machine Learning), 25 Aug 2025