In this article
Before you start
This page belongs to the machine learning (ML) system design interview path and applies its framing steps to time-series data.
Why interviewers ask this
A forecasting question tests whether a candidate protects the evaluation before building the model. Candidates report prompts such as forecasting bike pickups at one city station, forecasting monthly active users (MAU), or predicting the next few steps of a numeric series. Each one looks like a modelling task, and each hides the same trap: the future cannot be sampled at random.
A weak answer names a neural network and a train-test split. A strong answer states the decision the forecast feeds, the horizon, a baseline to beat and a way to score forecasts on later data only. A Staff-level answer adds per-horizon error, a scale-free metric across series, prediction intervals and what the system does when the world changes. Nothing here depends on one model family, which is why the question separates people who have shipped forecasts from people who have only fitted them.
Forecasting system from first principles
- Forecasting system
- A forecasting system is a pipeline that ingests a history of timestamped values, produces forecasts for a fixed horizon, and measures each forecast against actual values that arrive later, so that its error is known before anyone relies on it.
- Rolling-origin evaluation
- Rolling-origin evaluation is a way of scoring a forecasting method in which the cut between training and test data moves forward in time, and each test window always lies after its training data.
- Seasonal-naive forecast
- A seasonal-naive forecast is a baseline that predicts each future step as the value observed one full season earlier, such as the same weekday last week, with no fitted parameters.
- Scaled error
- A scaled error is a forecast error divided by the average error of a naive forecast on the training data, so it has no unit and can be compared across series of different size.
In plain words
Picture a shop owner who must order stock every Monday for the week ahead. She looks at past weeks, spots that Saturdays are busy, and orders to match. To test her habit, she pretends it is an old Monday, hides that week, and compares her order with what sold. The analogy breaks in one place. A shop owner learns from empty shelves, but a forecasting system must prove its error before it is trusted.
In one precise sentence, a forecasting system fits on the past, forecasts a horizon, and is judged only on values that came after the data it used.
Step 1: Frame the problem before choosing a model
State the decision, the unit, the horizon and the loss. A bike-share forecast might be pickups per station per hour for the next day, feeding a rebalancing plan. Each choice changes the design: granularity sets how noisy the target is, and the horizon sets how far ahead features must be known.
The loss matters too. The textbook Forecasting: Principles and Practice, §5.8 notes that a method minimising mean absolute error forecasts the median, while one minimising root mean squared error (RMSE) forecasts the mean. If stockouts hurt more than surplus, say so, and consider a quantile target instead.
Input: a business question. Output: a written target, horizon, unit and loss. You now know what "good" means, which the next step needs.
Step 2: Baselines come before any model
Every design starts with a naive forecast, which repeats the last value, and a seasonal-naive forecast, which repeats the value from one season earlier. They cost nothing to build and set the bar. The book describes skill scores, which measure accuracy relative to a benchmark. For seasonal data this page recommends the seasonal-naive forecast as that benchmark, and the book's beer-production table shows why: seasonal naive has a scaled error of 0.94 against 4.01 for naive.
Then add features. Useful ones are calendar fields, lagged values and drivers known in advance, such as holidays or a published schedule. A feature must exist at forecast time. Using tomorrow's actual weather to forecast tomorrow's demand leaks the answer, and the system fails in production.
Input: history and known-future drivers. Output: baseline scores plus a candidate feature set. A model that cannot beat the seasonal-naive forecast is not worth shipping.
Which models to try after the baseline
After the baseline, climb a short ladder and stop when the backtest stops improving. First come per-series statistical models such as exponential smoothing and ARIMA (autoregressive integrated moving average). Second is one global gradient-boosted-tree model trained across many series on lag and calendar features; the M5 accuracy competition results paper reviews the methods used in a large retail forecasting contest, and this page treats a global tree model as a strong candidate to test, not a rule. Third come deep sequence models, only when there are many related series and the rolling-origin backtest shows a gain over the cheaper models.
Whatever the model, choose a multi-step strategy. A direct strategy trains one model per horizon step. A recursive strategy trains a one-step model and feeds its forecasts back in as inputs, so errors can compound.
Input: baseline scores and features. Output: a ranked list of candidates scored on the same origins.
Step 3: Rolling-origin evaluation scores the forecast the way it will be used
Score forecasts on data not used for fitting. The book is direct: accuracy can only be determined on new data that were not used when fitting the model, and residuals on training data are not a reliable measure of true forecast error.
So build several origins, a method the book calls time series cross-validation. Fit on data up to an origin, forecast the horizon, score it, then move the origin forward and repeat. Each test window comes after its training data, which a random split cannot promise.
Report a scaled error. The same chapter explains that a scaled error below one comes from a better forecast than the average one-step naive forecast on training data, and above one is worse. That is the non-seasonal version of the scale. For seasonal data the book also defines a seasonal variant, which divides by the average gap between a value and the value one season earlier, and it is the one to use when the seasonal-naive forecast is your benchmark.
Step 4: Serve with intervals, reconciliation and monitoring
Production adds three jobs. First, ship a prediction interval, which is a range expected to contain the actual value with a stated probability, because a planner needs the spread as well as the point. Judge intervals by coverage, the share of actual values that land inside them, and by the quantile (pinball) loss, which the book covers in §5.9. Second, make related series agree. For store, region and total series, the book's chapter on hierarchical forecasts reports that forecasts reconciled with its MinT method are more accurate on average than the unreconciled base forecasts, though not for every series.
Third, monitor. A strong answer states that the live error is tracked against the seasonal-naive baseline, that a retraining trigger exists, and that late or missing data is flagged and never filled silently. These are design choices to state and justify, not facts every system shares.
Input: a chosen model and its scores. Output: forecasts with ranges, consistent totals and an alert when error drifts.
The architecture around the model
A forecasting system runs as a scheduled batch job that retrains and forecasts. It builds features from as-of snapshots, so each training row sees only what was known at that time, which keeps the serving features free of leakage. It writes the forecasts to a forecast store that the planner reads, so the planner never calls the model directly. Keep the job repeatable, so a rerun for the same origin writes the same forecasts, and log each forecast with its origin so the actual values can be matched to it later.
Step-by-step description
- Fix the decision, horizon and loss.
- Score naive and seasonal naive first.
- Roll the origin and score later data.
- Add intervals, reconciliation, monitoring.
- Frame the decision, horizon and loss first.
- A baseline comes before any model.
- Rolling-origin evaluation scores each forecast on later data.
- Serving adds prediction intervals, reconciliation and monitoring.
Frame the decision, horizon and loss first. A baseline comes before any model. Rolling-origin evaluation scores each forecast on later data. Serving adds prediction intervals, reconciliation and monitoring.
Knowing it is half the job. Saying it is the other half.
Interview Clozo is an AI interview copilot for real interview rounds: it hears the question and puts a strong answer on your screen, and you say it in your own words. See the features or how it works.
Start Free Trial →Worked example
The series below is synthetic and illustrative: 140 days of demand with a weekly pattern, a slow upward trend and seeded random noise.1 The horizon is seven days. Four origins, at days 112, 119, 126 and 133, give 28 scored forecasts.1 The scaling factor for each origin is the average one-step naive error on its training data, about 14.9 at the first origin.1
Four methods are compared: the naive forecast; the seasonal-naive forecast; the mean of the same weekday over the last four weeks; and the average of those last two methods.1
| Method | Mean absolute error (all 28 forecasts) | Scaled error |
|---|---|---|
| Naive | 24.61 | 1.644 |
| Seasonal naive | 4.21 | 0.282 |
| Four-week seasonal mean | 3.98 | 0.266 |
| Average of the two seasonal methods | 3.84 | 0.257 |
The plain naive forecast scores a scaled error of 1.644 and the seasonal-naive forecast 0.282.1 The naive value is above one because it forecasts a week ahead, while the scale uses one-step errors. Do not over-read the gap between 0.266 and 0.257: 28 illustrative forecasts cannot separate them.
Text description of the diagram
- The seasonal-naive forecast scores a scaled error of 0.282 and the plain naive forecast 1.644.
- The average of the two seasonal methods scores 0.257.
A real data set shows the same ordering effect. In the book's test-set table for monthly eating-out spending (read 2026-10-26), the combination forecast has a mean absolute scaled error (MASE) of 0.776, the ARIMA model 0.949 and the weakest single model 2.46. The book also reports that combining forecasts has led to increased accuracy. A simple average beat each of its parts.
Trade-offs and edge cases
The error measure changes the answer. The table lists four common ones and where each fails.
| Measure | What it rewards | Where it breaks |
|---|---|---|
| Mean absolute error (MAE) | Forecasts of the median (textbook §5.8) | Depends on the unit, so series cannot be compared |
| Root mean squared error (RMSE) | Forecasts of the mean (textbook §5.8, squared errors) | Harder to interpret |
| Mean absolute percentage error (MAPE) | Easy to read | Undefined or extreme if an actual is zero or near zero (textbook §5.8, percentage errors) |
| MASE | Comparable across series; below one beats the in-sample one-step naive | Needs training data for the scale |
To see the median-versus-mean point, take the illustrative values 1, 1, 1, 1 and 10. Their median is 1 and their mean 2.8. A constant forecast at the median has an absolute error of 1.8, against 2.88 at the mean; at the mean, root mean squared error is 3.6, against 4.025 at the median.1 Pick the loss that matches the decision.
Report error by horizon. The book notes that error increases as the horizon grows. In seasonal data the pattern can differ. Here every origin falls on the weekly low (pattern −20), and the last observed value sits on the second-highest day (+25), one day after the weekly peak (+30), so the naive error is 49.45 on day 1, smallest on day 6 (2.57) and 4.85 on day 7, where it equals the seasonal-naive error.1 A single average hides this.
Text description of the diagram
- The naive error is 49.45 on day 1 and 4.85 on day 7, where it equals the seasonal-naive error.
- Error depends on the horizon step.
Edge cases: intermittent demand with many zeros needs a scaled error, not a percentage one; very short histories favour the baseline; and a hierarchy of series needs reconciliation. For forecasting at scale across many series, a few simple methods per series, averaged, are a reasonable start, and a shared global model is an option to justify with your own backtest. For flagging odd points rather than forecasting them, see designing an anomaly-detection system. Related forecasts include estimated time of arrival (ETA) prediction and dynamic surge pricing.
How to answer it in an interview
How would you design a forecasting system?
I would start by fixing the decision, the horizon and the loss, for example demand for the next seven days. Before any model I would score a seasonal-naive baseline, because a model has to beat it. I would evaluate with a rolling origin, so every test point is later than its training data, and report a scaled error per horizon step. For many series I would average a few simple models and reconcile totals so stores add up to regions. In production I would ship prediction intervals and monitor error against the baseline.
The spoken structure: frame, baseline, rolling-origin evaluation, then serving and monitoring. If you have a minute more, name the trade-off between the loss you train on and the statistic you forecast.
Common mistakes
- "Training error is low, so the forecast is good." (wrong) Fit residuals understate true forecast error. (right) "I score on held-out data that is later than the training data."
- "I will split the data randomly into train and test." (wrong) Test points can precede training points, which is not how the model is used. (right) "I use rolling origins, each test window after its training data."
- "MAPE is the standard metric." (wrong) It breaks when actuals are zero. (right) "I use a scaled error such as MASE, and compare with the seasonal baseline."
- "The most complex model wins." (wrong) The book's table shows an average of models beating each single model. (right) "I try a simple baseline and an average before anything heavier."
- "I report one average error." (wrong) It hides how error changes by horizon step. (right) "I report error per horizon step."
Follow-up questions
How would you evaluate a forecasting model?
Use rolling origins, so each test window comes after its training data. Report a scaled error per horizon step next to the naive and seasonal-naive baselines. See also why pre-deployment metrics can mislead before a live test.
Why not use a random train-test split for a time series?
Accuracy is judged on data not used when fitting, and a random split lets the model train on points later than the ones it is tested on. That is not how the forecast will be used, so the score is too optimistic.
How would you forecast monthly active users?
Frame it as a monthly series with a stated horizon, start with naive and seasonal-naive baselines, and add drivers known ahead, such as launches or the calendar. Report intervals, because a planner needs the range.
How do you forecast many series at scale?
Score every series with a scaled error so results are comparable, start from baselines and an average of simple models, and reconcile series that must add up. The book reports that forecasts reconciled with its MinT method are more accurate on average than the unreconciled base forecasts, though not for every series.
Should you average several forecasting models?
Often yes. The book reports that combining forecasts has led to increased accuracy, and its table shows a combination beating each single model. Test it on your own rolling origins.
How would you detect anomalies in a multivariate time series where sensors have different latencies and missing values are common?
Align readings on event time and wait a per-sensor lateness window before scoring, so a slow sensor is not mistaken for a missing one. Feed missingness indicators to the model instead of imputing silently. Forecast each sensor, or model them jointly, and flag points whose residual leaves the prediction interval, backtesting on data replayed in the order it arrived. The anomaly-detection design page covers the full approach.
- Fix the decision, horizon and loss before choosing a model.
- Beat a seasonal-naive baseline first; it is the bar for every model.
- Score by rolling origin, on later data only, never on fit residuals.
- Use a scaled error, and report it per horizon step.
- Ship intervals, reconcile related series, and monitor against the baseline.
Sources
- 5.8 Evaluating point forecast accuracy, Forecasting: Principles and Practice (3rd ed.) — OTexts (Hyndman and Athanasopoulos), 1 Jan 2021
- 5.9 Evaluating distributional forecast accuracy, Forecasting: Principles and Practice (3rd ed.) — OTexts (Hyndman and Athanasopoulos), 1 Jan 2021
- 5.10 Time series cross-validation, Forecasting: Principles and Practice (3rd ed.) — OTexts (Hyndman and Athanasopoulos), 1 Jan 2021
- 13 Some practical forecasting issues, Forecasting: Principles and Practice (3rd ed.) — OTexts (Hyndman and Athanasopoulos), 1 Jan 2021
- 11 Forecasting hierarchical and grouped time series, Forecasting: Principles and Practice (3rd ed.) — OTexts (Hyndman and Athanasopoulos), 1 Jan 2021
- Another look at measures of forecast accuracy (the MASE paper) — International Journal of Forecasting 22(4) (Hyndman and Koehler), 1 Oct 2006
- M5 accuracy competition: Results, findings, and conclusions — International Journal of Forecasting 38(4) (Makridakis, Spiliotis, Assimakopoulos), 1 Oct 2022
