Design real-time fraud detection

In this article

1. Before you start
2. Why interviewers ask this
3. Real-time fraud detection from first principles
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one worked problem in the machine learning (ML) system design guide.

Why interviewers ask this

A fraud question packs most of the hard parts of applied machine learning into one prompt: a hard latency limit, a very rare positive class, features that must be fresh, labels that arrive late and an opponent who changes tactics. Interviewers use it to see whether you start from the decision and its costs or from a model name.

A weak answer picks an algorithm and reports accuracy. With fraud this rare (about 0.2% of transactions in the datasets of Dal Pozzolo et al., 2018), a model that approves everything is about 99.8% accurate and useless. A strong answer states the cost of each error, designs the streaming features and the latency budget, explains what happens when labels are late and says what the system does when a component fails. A typical prompt adds a number, such as a decision in under 200 milliseconds, concept drift and a full audit log. It suits system design rounds for payments and fintech roles.

Real-time fraud detection from first principles

Fraud scoring

Fraud scoring is the step in which a trained model assigns each payment request a risk score within the time the payment is waiting, so a decision system can approve, decline or send the payment to review.

Velocity feature

A velocity feature is a count or total of a card's, device's or account's recent activity over a short window, such as the number of attempts in the last ten minutes, kept in a fast store so it is ready at request time.

In plain words

Think of a shop assistant who must decide in a few seconds whether to hand over goods for a card. The assistant looks at the card, the buyer and what has happened at the till in the last few minutes. Most buyers are honest, so the assistant cannot interrogate everyone. A wrong approval costs the goods. A wrong refusal costs a sale. Fraud scoring automates that glance. The analogy breaks here: the assistant can ask for ID and judges one buyer at a time, while the system must decide from data alone, for thousands of payments a second.

In one precise sentence: a real-time fraud system turns a payment request into fresh features, scores it, applies a cost-based decision rule and logs everything, while a slower loop gathers late labels and retrains the model.

Step 1: Frame the decision and its costs

The input is the business goal and the two ways to be wrong. A false decline is a genuine payment that is refused; a chargeback is the reversal a card issuer forces when a cardholder disputes a payment; for an approved fraud the merchant loses the amount and pays a fee. The output is a decision rule. If a fraud costs 50 times a false decline, which is an illustrative ratio, the best rule declines a payment when its fraud probability exceeds C_decline / (C_decline + C_fraud) = 1 / (1 + 50) = 1 in 51, about 2%.¹

The reasoning is expected cost: decline when the probability times the fraud cost is higher than the genuine probability times the decline cost. It also tells you the model must output calibrated probabilities, so that a score of 0.02 really means about 2%. Stripe documents a score from 0 to 99 with default bands for elevated and high risk, so a threshold is a product setting (Stripe Radar documentation, read October 2026). What you now know: the threshold comes from costs.

Step 2: Build streaming features, especially velocity features

The input is the payment request plus history. Behaviour carries most of the signal: how often this card, device or email appeared recently, how far this amount is from the card's usual amount and whether the shipping and billing details match. Dal Pozzolo and colleagues describe the same idea as aggregated features, such as the average spend per week and the number of transactions in a day (Dal Pozzolo et al., 2018).

A database query per feature on every request does not fit a 100-millisecond budget, so keep a small state per card in a streaming store and update it as events arrive. A queue of timestamps per card, trimmed to the last ten minutes, answers "how many attempts in ten minutes" in a few operations per event, however long the history. In a toy run of 100,000 events over 2,000 cards, this took one twelfth as many operations (12.7 times fewer) as rescanning each card's history, with identical feature values.¹ Stripe says its model assesses more than 1,000 characteristics of a transaction (Stripe Engineering, March 2023).

Step 3: Handle labels that arrive weeks late

A fraud label is usually a dispute, which arrives after the cardholder spots the charge. Dal Pozzolo and colleagues describe two kinds of labels: a few feedbacks from investigators on recent alerts and a large volume of delayed labels that become available after several days. After a period without dispute, unreported transactions are treated as genuine (Dal Pozzolo et al., 2018).

That default creates a trap. Suppose confirmation time is exponential with a 14-day average, an illustrative assumption. The share of a row's fraud confirmed by age a days is then 1 − e^(−a/14). A row that is 3 days old shows only 19% of the fraud it will eventually show, a 7-day-old row 39%, a 14-day-old row 63% and a 30-day-old row 88%. Averaged over the last 7 days' rows, the share is 21%; rows aged 7 to 14 days average 52%.¹ A model trained on yesterday's data learns that recent behaviour is safe. The fixes are to exclude rows younger than a maturity cutoff, weight recent confirmed cases up and use investigator feedback for fresh signal.

Text description of the diagram

• With a 14-day average delay, a 3-day-old row shows 19% of its true fraud.
• A 30-day-old row shows 88% of its true fraud.

Step 4: Score, decide and plan for failure

The input is the feature vector; the output is a score, a decision and a log line. Stripe reports that its model makes its decision in less than 100 milliseconds (Stripe Engineering, March 2023). To fit it, split the budget by stage and read features in one batched call. In the example budget, 15 milliseconds go to network and gateway, 20 to feature reads, 25 to scoring, 10 to rules and 5 to hand-off of the log, with 25 of slack: 100 in total.¹

Fan-out, sending one request to several services in parallel, multiplies tail delays. A lookup's p99 latency is the time that 99 calls in 100 beat, so 1 call in 100 is slower. If 8 parallel lookups are slow independently, 1 − 0.99^8 = 7.73% of requests wait on at least one slow call.¹ Decide in advance what happens when scoring times out: fail open (approve), fail closed (decline) or fall back to a rules score. The answer depends on the cost ratio of Step 1. Log the features, model version and decision for every request.

Step 5: Retrain, roll out and watch for drift

Concept drift is a change over time in what fraud looks like or how customers behave, so a model's accuracy decays. Dal Pozzolo and colleagues name concept drift, class imbalance and verification latency as the three challenges of the problem (Dal Pozzolo et al., 2018). Retrain on a schedule from matured labels, and compare them on the same matured hold-out window for fraud metrics, and on shadow-scored live traffic for decline rate and latency, before the new model takes over.

Roll out in shadow mode first, scoring live traffic without acting, then to a small share of traffic, with a one-step rollback. Google's Rules of Machine Learning says to save the features used at serving time and to measure training-serving skew by applying the model to a training example and to the same example at serving; the results should match (Rules of ML, rules 29 and 37, updated August 2025). Keep a small exploration slice that is never blocked, so you can still learn what the blocked traffic would have done.

1 · Frame costs2 · Stream features3 · Mature labels4 · Score and decide5 · Retrain and releaseReplay

Step-by-step description

1. Cost ratio sets the decline threshold.
2. Windowed state is ready at request time.
3. Late labels are aged before training.
4. Model and rules decide inside the budget.
5. Shadow mode, then a small share, then all.

• The decline threshold comes from the cost of each error.
• Velocity features are kept as streaming state and read in one batched call.
• Fraud labels arrive weeks late, so training rows must be matured.
• Retraining is released in shadow mode before it takes live traffic.

Worked example

The worked example runs four calculations on illustrative inputs; the code and its saved output are named in the note.¹

Why accuracy misleads. Take a model that catches 80% of fraud and wrongly flags 1% of genuine payments. At a fraud rate of 1 in 100 its alerts are right 44.7% of the time. At 1 in 1,000 they are right 7.4% of the time, which is 13.5 alerts per real fraud, and at 1 in 10,000 only 0.8%.¹ Rarity, not model quality, drives the false-decline load: at 1 in 10,000 the same model is about 99.0% accurate (0.8 × 0.0001 + 0.99 × 0.9999), yet only 0.8% of its alerts are right.¹

The threshold. A toy run scores 500,000 payments whose true fraud probabilities are known, so the model is perfectly calibrated. It contains 875 frauds, a rate of 0.175%. With a cost ratio of 50 to 1, approving everything costs 43,750 units. Declining above the cost-based threshold of 0.0196 costs 14,247 units, catches 732 of the 875 frauds and wrongly declines 7,097 genuine payments.¹

Text description of the diagram

• Approving everything costs 43,750 units in the toy run.
• Declining above the cost-based threshold of 0.0196 costs 14,247 units.

The velocity feature and the latency fan-out are in Steps 2 and 4: 12.7 times fewer operations for the streaming window, and 7.73% of requests slowed when 8 lookups run in parallel.¹

Trade-offs and edge cases

A fraud system trades lost sales against lost goods, and each design choice moves that balance.

• Rules or model. Rules are fast to ship and explain; a model finds combinations of weak signals. Both Stripe Radar and the system Dal Pozzolo et al., 2018 describe use both, with rules for known attacks and the model for the rest; the latter describe blocking rules that run before the data-driven model.
• Fail open or closed. Failing closed loses sales when the scorer is down; failing open invites an attack on the outage. A common compromise is a cheap fallback score with a lower decline rate.
• Review queue size. Investigators can check only a few alerts a day, so alerts compete for attention and the alert budget caps recall however good the model is. Dal Pozzolo et al., 2018 therefore measure precision among the top k alerts.
• Feedback loops. Blocked payments never produce labels. Keep an exploration slice; Stripe notes it modifies the reported score for a small subset of payments to measure model performance (Stripe Radar documentation, read October 2026).
• Adversaries and drift. Fraudsters adapt to the model, so recency weighting and fast retraining matter as much as the algorithm; see distribution shift and anomaly detection in time series.
• Graph signals. Rings of linked cards, devices and addresses are hard to see in per-payment features; a graph feature job can add them as another streaming input.

For the neighbouring designs see the payment system design, the feature store walkthrough and machine learning operations (MLOps). The base-rate arithmetic is Bayes' theorem at work.

How to answer it in an interview

The spoken structure: costs, features, budget, labels, release.

Common mistakes

• "I would maximise accuracy." (wrong) Approving everything is about 99.8% accurate when 0.2% of payments are fraud. (right) "I would minimise expected cost, which sets the threshold."
• "I would query the database for recent activity on each request." (wrong) That adds a slow call per feature to a tight budget. (right) "I would keep streaming windowed state and read it in one batched call."
• "I would train on last week's data to catch new fraud." (wrong) Last week's labels are mostly missing, so fraud looks rarer. (right) "I would train on matured rows and add investigator feedback for recent cases."
• "If the model is down we approve everything." (wrong) The choice ignores cost and invites abuse. (right) "I would fall back to a rules score, and choose open or closed from the cost ratio."

Follow-up questions

How would you design a real-time fraud scoring system that decides in about 200 milliseconds, handles concept drift and keeps an audit log?

Frame the cost ratio, build streaming features, score with a calibrated model inside a stage-by-stage latency budget and log the features, model version and decision per request. Handle drift with scheduled retraining from matured labels and staged releases.

Given raw transaction logs, what features would you engineer for a real-time fraud model?

Behavioural aggregates per card, device and account over several windows, the gap between this amount and the usual amount, and consistency checks between shipping, billing and device details. Dal Pozzolo and colleagues give weekly average spend and daily transaction counts as examples.

How do you compute velocity features in real time without hitting the database on every transaction?

Keep a windowed counter or timestamp queue per key in a streaming store, update it as each event arrives and trim expired entries. In the toy run this took 12.7 times fewer operations than rescanning history, with no mismatched values.¹ The feature store walkthrough covers where such state lives.

How do you handle model retraining and deployment without downtime?

Train a candidate on matured labels, score live traffic with it in shadow mode, compare it with the incumbent, then shift a small share of traffic and keep rollback to the previous version one step away. Google's Rules of Machine Learning (updated August 2025) say to measure training-serving skew, and that applies at each stage.

What changes if a fraud costs only 5 times a false decline?

The threshold becomes 1 / (1 + 5), about 16.7%, so the system declines far fewer payments and accepts more fraud.¹

What is your fallback if the model service is down?

Use a simple rules-based score with conservative limits, and choose open or closed from the Step 1 cost ratio. Alert on the fallback rate, since it silently changes both losses and customer experience.

Sources

1. Credit Card Fraud Detection: A Realistic Modeling and a Novel Learning Strategy (IEEE Transactions on Neural Networks and Learning Systems 29(8)) — Dal Pozzolo, Boracchi, Caelen, Alippi and Bontempi (IEEE), 1 Aug 2018
2. How we built it: Stripe Radar — Stripe, 29 Mar 2023
3. Transaction risk prevention (Stripe Radar documentation; undated page, read 2026-10-24) — Stripe, 24 Oct 2026
4. Rules of Machine Learning: Best Practices for ML Engineering — Google for Developers, 25 Aug 2025