How do you design churn, LTV and credit-risk prediction?

In this article

1. Before you start
2. Why interviewers ask this
3. Churn, LTV and credit-risk prediction from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one design in the machine learning (ML) system design interview guide. Neighboring designs that reuse its ideas are real-time fraud detection and anomaly detection.

Why interviewers ask this

A model that scores well on a metric and still loses money is the failure this question is built to expose. Candidates have reported four versions of it: predict whether a rider cancels after requesting a ride, predict whether a driver cancels a trip, approve installment loans for new users with no history, and estimate what a listing will earn over the next 12 months. A churn-system design for a payments company is also reported.

A weak answer picks an algorithm and a metric. A strong answer starts with the decision the score will drive, defines the label and its time window, and says what happens to people the model never sees, such as refused applicants. A Staff-level answer shows that the score becomes money through calibration and a cost or value formula, and names what to monitor once the model's own decisions change the data.

Churn, LTV and credit-risk prediction from first principles

LTV

LTV is a forecast of the total margin a customer will bring over a stated period, such as the next twelve months, built from how often they buy, how much each purchase is worth and how likely they are to leave.

Censored label

A censored label is a label that is not known yet because the event has not happened by the time you look, such as a user who has not canceled so far but may cancel next month.

Uplift

Uplift is the change in an outcome caused by an action, such as the extra share of customers who stay because they received an offer, compared with what they would have done without it.

In plain words

Think of a weather forecaster. A forecaster who says "70% chance of rain" is useful only if it rains on about 7 of 10 such days. And knowing it will rain does not tell you whether to sell umbrellas; that depends on price and stock. The analogy breaks in one place: a customer reacts to your action, while the weather ignores your umbrella.

In one precise sentence, these three systems estimate a probability or a value for each customer, check that the estimate is honest, and combine it with a cost to choose an action. Churn is the event that a customer stops using a service. Expected loss is the average money a lender loses on a loan. It equals the probability of default, times the share lost if it defaults, times the amount owed at that moment.

Step 1: Name the decision, the label and the window

Start from the action: send an offer, rank listings, approve or refuse a loan. The label must match that action. For churn, write the exact rule, for example no activity for 30 days, and the horizon, for example leaving within the next 30 days. For credit, define default and the window over which you wait for it.

Time matters because outcomes are not all in yet. The lifelines survival guide describes users as joining ("birth") and leaving ("death"), and notes that at analysis time not every death has happened, which is censoring. A user who has not left yet is a censored label, not a safe customer. Input: events with dates. Output: one label per customer and an observation window. You now know what the model must predict and for whom.

Step 2: Turn the score into money, with calibration and uplift

A raw score becomes a decision only through money. First, check calibration. The scikit-learn guide says a well-calibrated classifier scores about 0.8 on cases of which about 80% are positive, and warns that a lower Brier loss does not necessarily mean better calibration. High AUC can hide wrong probabilities, so measure calibration separately and fix it with a calibration step on held-out data.

Then apply a formula. For credit, expected loss is the probability of default times the loss given default times the exposure at default, the three terms of the Basel framework's credit-risk chapters. For churn, profit per customer contacted is churn probability times save rate times value, minus the offer cost. Target the customers with the best profit, not the highest risk. For LTV, forecast purchases and value per purchase, then discount. The worked example computes all three.

Step 3: Train on a time split and check the right metrics

Split by time: train on older customers or periods, validate on newer ones. A random split lets the model see the future and usually inflates the metrics. Build features only from data available at the scoring moment, because a feature recorded after the outcome, such as a support call made while canceling, is leakage.

Churn and default are rare, so accuracy is useless: a model that always says "stays" is almost always right. Report precision and recall, the area under the precision-recall curve, and calibration, and choose the threshold from the cost of each error rather than from 0.5. Start with a simple model such as logistic regression or gradient-boosted trees, and add survival or probabilistic models where time matters. Input: features at the scoring time. Output: calibrated probabilities.

Step 4: Monitor, and plan for the cold start

Once the model drives decisions, it changes its own data. If you refuse every low score, you never see whether those applicants would have repaid, a bias called selection bias. Reserve a small, controlled exploration slice, or use reject-inference techniques with care, so the model keeps learning from people it would refuse. For churn, keep a random holdout that receives no offer, because it is the only clean measure of uplift.

For a brand-new product with no history, begin with application data, simple rules and low exposure limits, then widen as outcomes arrive. Monitor calibration by score bucket, input drift and the base rate each week. In the United States, Regulation B section 1002.9 requires a lender to notify an applicant of an adverse decision and to give the principal reasons or say how to request them. Other markets have similar rules, so check the explainability rule where you operate before choosing a black-box model.

1 · Decision and label2 · Money3 · Train4 · MonitorReplay

Step-by-step description

1. Name the action, the label and the window.
2. Calibrate the score and rank by profit.
3. Split by time and check the right metrics.
4. Keep a holdout and plan the cold start.

• The design starts from the decision and the label with its window.
• The score becomes money through calibration and uplift.
• Training uses a time split, and monitoring keeps a holdout.

Worked example

All inputs here are illustrative, and every result was computed in a script.¹ First, a lender scores one loan: probability of default 0.04, loss given default 0.60, exposure 30,000 units. Expected loss is 0.04 × 0.60 × 30,000 = 720 units. If the true default rate is double, 0.08, the real expected loss is 1,440 units, so a model with good AUC but probabilities that are half of reality under-prices every loan.¹ In an illustrative calibration table, scores of 0.02, 0.05, 0.10 and 0.20 correspond to observed default rates of 0.04, 0.08, 0.15 and 0.30.

Second, censoring. Take a cohort of 1,000 people (illustrative): 300 observed for 2 months with 9 churned, 300 for 4 months with 24 churned, and 400 for 6 months with 60 churned. Counting the 93 churners over all 1,000 people gives 9.3%. But the users were watched for different lengths: exposure is 3,963 user-months, assuming churners left halfway through their window, so the monthly hazard is 2.347% and six-month churn is 13.28%.¹ The naive 9.3% therefore understates six-month churn, because most users were watched for less than six months. The hazard figure also assumes the monthly hazard is constant.

Third, whom to contact. Each segment below holds 10,000 people (illustrative), and an offer costs 15 units.

Ranking by churn probability picks the high-risk, hard-to-save segment; ranking by profit picks the medium-risk, easy-to-save, high-value one.¹ Uplift modeling sorts customers into four classic groups: persuadables, whom an offer moves to stay; sure things, who stay anyway; lost causes, who leave regardless; and sleeping dogs, whom an offer pushes to leave. Sleeping dogs are the real reason a no-offer holdout matters.

Fourth, LTV. With a margin of 10 per month, 5% monthly churn and 1% monthly discounting, month 1 adds 9.90 and month 12 adds 5.05. The 12-month value is 86.74, against 120 if nobody left and nothing were discounted.¹ The probabilistic route is the beta-geometric negative binomial model of Fader, Hardie and Lee (2005), which forecasts repeat purchases from frequency and recency, and the Gamma-Gamma model of Fader and Hardie (2013) values each purchase.

Text description of the diagram

• Month 1 adds 9.90 and month 12 adds 5.05.
• The 12-month value is 86.74 against 120 if nobody left.

Trade-offs and edge cases

The three problems share a design but not a label. Churn predicts who leaves and risks treating active users as safe; LTV predicts value and risks being a plain regression on past revenue that ignores customers leaving; credit risk predicts who will not repay and sees outcomes only for people it approved. Each has its own label window and its own trap.

Text description of the diagram

• Churn predicts who leaves, LTV predicts value and credit risk predicts who will not repay.
• Each has its own label window and its own trap.

Edge cases. Contractual churn (a subscription that ends on a date) is clean; in non-contractual settings such as rides or marketplaces you must choose an inactivity rule, and the rule changes the base rate. A short label window gives fast feedback but misses slow leavers; a long one waits months for labels. Where actions have prices, as in dynamic pricing, the model and the policy interact, so test the policy, not just the score; the guide on metrics that looked great before an A/B test covers why. Reported practice differs on how far to trust reject inference, so treat it as a hypothesis and verify it with a small exploration slice.

How to answer it in an interview

The spoken structure: decision, label and window, time split, calibration, money formula, then the holdout and monitoring.

Common mistakes

• "Users who have not churned are negatives." (wrong) They may churn later. (right) "I treat them as censored and use a fixed horizon or a survival model."
• "We got a high AUC, so the probabilities are usable." (wrong) AUC measures ranking only. (right) "I also check calibration by score bucket before using the probability in a price or a loss."
• "Contact the customers with the highest churn probability." (wrong) Some will leave anyway and some cannot be saved. (right) "I target the best expected profit, using uplift from a holdout."
• "Train on approved applicants and ship." (wrong) Refused applicants have no outcome, so the data is biased. (right) "I add a controlled exploration slice and track the bias."
• "LTV is a regression on past revenue." (wrong) It ignores who left. (right) "I model purchases, value and survival together."

Follow-up questions

How do you design a customer churn prediction system?

Define churn with a rule and a horizon, build features available at scoring time, train on a time split, calibrate, and rank customers by expected profit of the action rather than by risk. Keep a random no-offer holdout to measure uplift and monitor drift weekly.

How would you predict whether a user will cancel a ride after requesting it?

Score at request time using only what is known then: pickup and drop location, time, estimated arrival time, price and the rider's history. The label is a cancel before pickup. Useful actions are better matching or a clearer arrival estimate, as in arrival-time prediction, so judge the model by how much it improves those decisions.

How would you build a model to predict the probability of a driver canceling a trip?

Treat it as the supply-side twin: score at dispatch, with the driver's distance to pickup, recent cancels, trip length and expected earnings. Leakage is the main risk, because features such as a call to the rider happen after the trigger. Use the probability to re-rank dispatch candidates.

How do you design credit risk modeling for EMI approvals with no user data and no bank information?

For an EMI (equated monthly installment) plan, begin with what you can lawfully collect at application, such as device and order signals, and simple rules with small limits. Approve a controlled slice beyond what the model would approve, so repayment outcomes accumulate, then retrain with a time split. Keep decisions explainable and check the lending rules where you operate.

How do you estimate the value of a listing over the next 12 months?

Forecast bookings and revenue per booking per month, apply the chance the listing stays active, discount, and sum, as in the worked example. Use the number for ranking, as in search ranking, and for host decisions, and validate it on a later period.

Why can a high AUC still give bad credit decisions?

AUC only checks that riskier applicants rank above safer ones. If probabilities are half the true rate, every price and loss estimate is wrong even if the ranking is perfect.

Sources

1. Probability calibration (scikit-learn user guide, section 1.16) — scikit-learn developers, 26 Oct 2026
2. Introduction to survival analysis (lifelines documentation) — lifelines documentation (Cameron Davidson-Pilon and contributors), 26 Oct 2026
3. 'Counting Your Customers' the Easy Way: An Alternative to the Pareto/NBD Model — Marketing Science 24(2) (Fader, Hardie and Lee), 1 May 2005
4. The Gamma-Gamma Model of Monetary Value — Fader and Hardie (technical note), 1 Feb 2013
5. Regulation B, section 1002.9: Notifications — Consumer Financial Protection Bureau, 26 Oct 2026
6. Basel framework, CRE31: IRB approach, risk weight functions — Bank for International Settlements, 26 Oct 2026