LSTM vs ARIMA forecasting: which should you use?

In this article

1. Before you start
2. Why interviewers ask this
3. What each one is
4. Worked example
5. Trade-offs and edge cases
6. How to answer it in an interview
7. Common mistakes
8. Follow-up questions
9. Sources

Before you start

This page is one step of the deep learning path.

Why interviewers ask this

The cost of a wrong choice is the reason this comparison comes up: a team that trains a neural network where a few coefficients would do spends more compute for a worse forecast. In a forecasting or data-science discussion, an interviewer uses LSTM versus ARIMA to see whether a candidate picks a model from the data at hand instead of from fashion.

A weak answer says "deep learning is more powerful, so use an LSTM." A strong answer asks how many series there are, how long they are and whether trend and seasonality are stable, then names a statistical baseline to beat. A Staff-level answer adds the competition evidence below and proposes a hybrid or a combination when neither model clearly wins. The deep learning interview questions page lists neighboring questions.

What each one is

ARIMA model

An ARIMA model is a linear forecasting model that expresses each value of a series as a combination of its own recent past values and recent forecast errors, after the series has been differenced enough to remove trend. It is written ARIMA(p, d, q) and is normally fitted to one series at a time.

Differencing

Differencing is the step of replacing each observation with its change from the previous observation. It removes a steady trend and is the usual way to turn a non-stationary series into a stationary one before fitting an ARIMA model.

Cross-learning

Cross-learning is training one forecasting model on many related time series together, so that patterns found in some series help forecast the others. It is the opposite of fitting a separate model to each series, and it is the setting in which neural networks have performed best.

ARIMA in plain words

An ARIMA model forecasts a series from its own recent past after removing any trend. Hyndman and Athanasopoulos define three orders: p is the order of the autoregressive part, d is the degree of first differencing, and q is the order of the moving average part. In plain terms, p counts how many past values feed the forecast, q counts how many past forecast errors feed it, and d counts how many times the series is differenced first.

The autoregressive moving average (ARMA) part of the model needs a stationary series, and d differencing steps make the series stationary. A stationary series is one whose statistical properties do not depend on the time at which it is observed, so a series with trend or seasonality is not stationary. One fix is to compute the differences between consecutive observations.

Think of ARIMA as a driver who reads only the last few mile markers and the last few steering corrections. The analogy breaks at a sudden new junction: ARIMA is linear and weights only a few recent values and errors directly.

LSTM in plain words

An LSTM is a recurrent neural network (RNN) whose memory cell can carry information across many time steps. Zhang et al. describe each ordinary recurrent node replaced by a memory cell with an internal state, a node with a self-connected recurrent edge of fixed weight 1. Hochreiter and Schmidhuber (1997) report that an LSTM can bridge time lags in excess of 1000 steps in their tests, using multiplicative gate units that learn to open and close access to the error flow. Those results come from artificial tasks, not from forecasting benchmarks.

The cell exists to ease vanishing gradients. Gradient clipping helps with exploding gradients, but Zhang et al. note that vanishing gradients need a more elaborate solution; see exploding gradients. The gates use activation functions, and every gate has its own weights, so an LSTM has far more parameters than an ARIMA model, as the worked example shows. For the cell in detail, read LSTM explained.

Cross-learning: where an LSTM earns its place

Cross-learning is the reason an LSTM can beat ARIMA: one network sees many series and shares what it learns. The M4 competition asked for forecasts of 100 thousand time series. Petropoulos et al. report that machine learning approaches focused on each series independently performed poorly against statistical benchmarks, while the best two submissions used neural network and machine learning algorithms to utilize cross-learning.

The winner was a hybrid. Makridakis et al. (2018) call a hybrid of statistical and machine learning features the biggest surprise, and the winning entry combined an LSTM with exponential smoothing (Smyl, 2020). Hewamalage et al. write that RNNs became competitive forecasting methods, most notably through that winning method. An LSTM wins when it can learn across many series, not when it is fitted to one.

Worked example

A small series shows how little ARIMA needs and how much an LSTM carries. The illustrative series has 8 monthly values: 10, 12, 15, 17, 20, 22, 25 and 27. Differencing once gives 7 changes: 2, 3, 2, 3, 2, 3 and 2.¹

Fit ARIMA(1, 1, 0), meaning one autoregressive term, one difference and no moving average term, with no constant. Each change is modeled as the coefficient times the previous change. Least squares gives a numerator of 36 (the six products of neighboring changes) and a denominator of 39 (the sum of squared earlier changes), so the coefficient is 36 / 39 = 0.9231.¹ The last change was 2, so the next forecast change is 0.9231 × 2 = 1.8462, and the forecast value is 27 + 1.8462 = 28.8462. The step after that adds 1.7041 and reaches 30.5503.¹

Because the coefficient is below one and there is no constant, each forecast change is smaller than the last, so growth fades. Adding a constant, a drift term, is the usual fix when the trend should continue.

An LSTM layer with h units and x inputs has 4 × (h × (h + x) + h) parameters: four blocks (the input, forget and output gates of the modern LSTM, plus the candidate memory; the 1997 cell had no forget gate), each with a weight matrix over the units and inputs plus one bias vector (some libraries keep two).¹

With 32 units and one input that is 4,352 weights, set against 8 observations. Fitted to this series alone, the LSTM has far more freedom than data, which is why single-series fits favor ARIMA. Real data point the same way: in the M3 study, the machine learning group, which included an LSTM fitted to each series on its own, was dominated by the statistical methods.

Trade-offs and edge cases

ARIMA is fitted to one series at a time, while an LSTM can be trained on many series at once. The table collects the evidence from forecasting studies.

The M3 study held out the last 18 observations of each series (Makridakis et al., 2018) and fitted on the rest. Besides eight machine learning methods, it tested two more that had become popular in recent years, a recurrent network and an LSTM, and each was fitted to one series at a time. The machine learning group, those two included, was dominated by the statistical methods. This is the closest single-series test of an LSTM on this page, and it fits the cross-learning explanation.

The state of play has moved since M4. In the M5 competition, gradient-boosted trees trained across series dominated (Makridakis, Spiliotis and Assimakopoulos, 2022). A pure deep model built on neural basis expansion was reported by its authors to beat the M4 winner (Oreshkin et al.). Pretrained time-series foundation models such as Chronos and TimesFM forecast new series without per-series training. So an LSTM is no longer the default neural forecaster; cross-learning, not the LSTM cell itself, is what wins.

Text description of the diagram

• ARIMA is fitted to one series at a time.
• An LSTM can be trained on many series at once.
• An LSTM needs far more parameters than ARIMA.

The table compares the models. ARIMA is linear in past values and errors, and an LSTM is nonlinear with gated memory. ARIMA handles trend by differencing (the d order) and seasonality by seasonal differencing in a seasonal ARIMA model. An LSTM still needs its trend handled, for example by detrending or scaling each series first; Makridakis et al. (2018) reports that the studies it reviews considered first differences for removing the trend component. For seasonality, Hewamalage et al. say RNNs can model it directly if the series share homogeneous seasonal patterns, and otherwise recommend a deseasonalization step. An LSTM needs far more parameters than ARIMA and, per the M3 study, considerably more computation.

Edge cases follow from this. For one short series, start with ARIMA or exponential smoothing. For many related series, such as thousands of product sales histories, a global LSTM model is worth testing, ideally inside a hybrid, and a combination of a few methods is often the safest. ARIMA gives model-based prediction intervals; an LSTM usually needs an added technique, such as training for quantiles, to give an uncertainty range. Compare any LSTM with a statistical baseline before you trust it.

Text description of the diagram

• Many related series favor a global LSTM model.
• Compare any LSTM with a statistical baseline.

How to answer it in an interview

The spoken structure: name the baseline, give the reason for it, say when the LSTM wins, then cite the evidence and the check.

Common mistakes

• "Deep learning always beats ARIMA." (wrong) The M3 and M4 evidence shows pure machine learning often lost to statistical methods and combinations. (right) "A neural network wins when it learns across many series, and I check it against a baseline."
• "It is statistics against machine learning." (wrong) The M4 winner was a hybrid that blended both. (right) "The best entries combined statistical and neural parts."
• "ARIMA needs no preprocessing." (wrong) The ARMA part assumes a stationary series, and trend or seasonality breaks that. (right) "I difference the series, seasonally if needed, before fitting."
• "An LSTM removes the need for preprocessing." (wrong) RNNs model seasonality directly only when patterns are homogeneous across series, and trend still needs handling. (right) "I detrend or scale each series, and deseasonalize if seasonal patterns differ across series."

Follow-up questions

Is an LSTM better than ARIMA for time series forecasting?

Not by default. On the 1045 monthly M3 series the statistical methods beat the machine learning methods at every horizon, including an LSTM fitted to each series alone, and in M4 the winner was a hybrid. An LSTM is better when it can learn across many related series.

Is an LSTM still the best neural model for forecasting?

No. Since M4, gradient-boosted trees trained across series dominated M5, a pure deep model was reported to beat the M4 winner, and pretrained foundation models such as Chronos and TimesFM forecast without per-series training. The common thread is learning across series, not the LSTM cell.

When should you use ARIMA instead of an LSTM?

Use ARIMA when you have one or a few short series, need a cheap and explainable model, or want model-based prediction intervals. It needs a handful of coefficients, where even a small LSTM layer has hundreds to thousands of weights.

Does an LSTM need stationary data?

The architecture does not require it, but trend and seasonality still matter. Hewamalage et al. say RNNs model seasonality directly only when the series share homogeneous seasonal patterns, and recommend deseasonalizing otherwise.

Can you combine ARIMA and an LSTM?

Yes, and the evidence favors it. In M4, 12 of the 17 most accurate methods were combinations, and the winner joined an LSTM with exponential smoothing. A simple version averages the forecasts of both models.

How do you compare two forecasting models fairly?

Hold out the end of each series, as the M3 study did with its last 18 observations, fit on the earlier part, and score both models on the same held-out horizon. Use rolling-origin evaluation, which repeats the split at several forecast origins, and a scale-free error measure such as mean absolute scaled error (MASE); Hyndman and Athanasopoulos cover both. Include a statistical baseline and report compute cost as well as error. The probability and statistics guide covers the underlying statistics.

Sources

1. Forecasting: Principles and Practice (3rd ed.), chapter 9: ARIMA models — OTexts (Hyndman and Athanasopoulos), 3rd edition, 1 May 2021
2. Forecasting: Principles and Practice (3rd ed.), section 5.8: evaluating point forecast accuracy — OTexts (Hyndman and Athanasopoulos), 3rd edition, 1 May 2021
3. Long Short-Term Memory — Neural Computation 9(8) (Hochreiter and Schmidhuber), 1 Nov 1997
4. Dive into Deep Learning, section 10.1: Long Short-Term Memory (arXiv 2106.11342) — arXiv (Zhang, Lipton, Li, Smola), 21 Jun 2021
5. Statistical and Machine Learning forecasting methods: Concerns and ways forward — PLOS ONE 13(3) (Makridakis, Spiliotis, Assimakopoulos), 27 Mar 2018
6. The M4 Competition: Results, findings, conclusion and way forward — International Journal of Forecasting 34(4) (Makridakis, Spiliotis, Assimakopoulos), 1 Oct 2018
7. Forecasting: theory and practice (arXiv 2012.03854) — arXiv (Petropoulos et al.), 1 Dec 2020
8. Recurrent Neural Networks for time series forecasting: current status and future directions (arXiv 1909.00590) — International Journal of Forecasting 37(1) (Hewamalage, Bergmeir, Bandara), 1 Jan 2021
9. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting — International Journal of Forecasting 36(1) (Smyl), 1 Jan 2020
10. M5 accuracy competition: Results, findings, and conclusions — International Journal of Forecasting 38(4) (Makridakis, Spiliotis, Assimakopoulos), 1 Oct 2022
11. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (arXiv 1905.10437) — arXiv (Oreshkin et al.), 24 May 2019
12. Chronos: Learning the Language of Time Series (arXiv 2403.07815) — arXiv (Ansari et al.), 12 Mar 2024
13. A decoder-only foundation model for time-series forecasting (arXiv 2310.10688) — arXiv (Das et al.), 14 Oct 2023