# Do We Really Need Deep Learning Models for Time Series Forecasting?

arXiv: https://arxiv.org/abs/2101.02118 · HTML: https://mlsanity.com/paper/2101.02118/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 6 Jan 2021.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction](https://mlsanity.com/paper/1704.02971/) (2017)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [GluonTS: Probabilistic Time Series Models in Python](https://mlsanity.com/paper/1906.05264/) (2019)
- [Deep Air Quality Forecasting Using Hybrid Deep Learning Framework](https://mlsanity.com/paper/1812.04783/) (2018)
- [Autoregressive Convolutional Neural Networks for Asynchronous Time Series](https://mlsanity.com/paper/1703.04122/) (2017)
- [Deep Multi-Output Forecasting: Learning to Accurately Predict Blood Glucose Trajectories](https://mlsanity.com/paper/1806.05357/) (2018)

## Built on by (most cited first, 30 of 45)

- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2306.10347/) (2023)
- [The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2304.05206/) (2023)
- [Improved Online Conformal Prediction via Strongly Adaptive Online Learning](https://mlsanity.com/paper/2302.07869/) (2023)
- [ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons](https://mlsanity.com/paper/2310.07446/) (2023)
- [Model scale versus domain knowledge in statistical forecasting of chaotic systems](https://mlsanity.com/paper/2303.08011/) (2023)
- [PatchAD: A Lightweight Patch-based MLP-Mixer for Time Series Anomaly Detection](https://mlsanity.com/paper/2401.09793/) (2024)
- [Deep Contrastive One-Class Time Series Anomaly Detection](https://mlsanity.com/paper/2207.01472/) (2022)
- [Does your graph need a confidence boost? Convergent boosted smoothing on graphs with tabular node features](https://mlsanity.com/paper/2110.13413/) (2021)
- [Monitoring Model Deterioration with Explainable Uncertainty Estimation via Non-parametric Bootstrap](https://mlsanity.com/paper/2201.11676/) (2022)
- [Federated Boosted Decision Trees with Differential Privacy](https://mlsanity.com/paper/2210.02910/) (2022)
- [A Comprehensive Survey on Generative Diffusion Models for Structured Data](https://mlsanity.com/paper/2306.04139/) (2023)
- [Explanation Shift: Detecting distribution shifts on tabular data via the explanation space](https://mlsanity.com/paper/2210.12369/) (2022)
- [Explanation Shift: How Did the Distribution Shift Impact the Model?](https://mlsanity.com/paper/2303.08081/) (2023)
- [Time Series Analysis and Modeling to Forecast: a Survey](https://mlsanity.com/paper/2104.00164/) (2021)
- [Explaining Exchange Rate Forecasts with Macroeconomic Fundamentals Using Interpretive Machine Learning](https://mlsanity.com/paper/2303.16149/) (2023)
- [Long-term series forecasting with Query Selector -- efficient model of sparse attention](https://mlsanity.com/paper/2107.08687/) (2021)
- [Forecasting Market Prices using DL with Data Augmentation and Meta-learning: ARIMA still wins!](https://mlsanity.com/paper/2110.10233/) (2021)
- [Gradient boosting machines and careful pre-processing work best: ASHRAE Great Energy Predictor III lessons learned](https://mlsanity.com/paper/2202.02898/) (2022)
- [Beyond Demographic Parity: Redefining Equal Treatment](https://mlsanity.com/paper/2303.08040/) (2023)
- [Maximally Machine-Learnable Portfolios](https://mlsanity.com/paper/2306.05568/) (2023)
- [Contextualizing MLP-Mixers Spatiotemporally for Urban Data Forecast at Scale](https://mlsanity.com/paper/2307.01482/) (2023)
- [TimeTuner: Diagnosing Time Representations for Time-Series Forecasting with Counterfactual Explanations](https://mlsanity.com/paper/2307.09916/) (2023)
- [A Distance Correlation-Based Approach to Characterize the Effectiveness of Recurrent Neural Networks for Time Series Forecasting](https://mlsanity.com/paper/2307.15830/) (2023)
- [Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting](https://mlsanity.com/paper/2407.19784/) (2024)
- [Interpretable Deep Learning for Forecasting Online Advertising Costs: Insights from the Competitive Bidding Landscape](https://mlsanity.com/paper/2302.05762/) (2023)
- [TSPP: A Unified Benchmarking Tool for Time-series Forecasting](https://mlsanity.com/paper/2312.17100/) (2023)
- [An Efficient and Generalizable Symbolic Regression Method for Time Series Analysis](https://mlsanity.com/paper/2409.03986/) (2024)
- [Deep incremental learning models for financial temporal tabular datasets with distribution shifts](https://mlsanity.com/paper/2303.07925/) (2023)

## Abstract (arXiv)

Time series forecasting is a crucial task in machine learning, as it has a wide range of applications including but not limited to forecasting electricity consumption, traffic, and air quality. Traditional forecasting models rely on rolling averages, vector auto-regression and auto-regressive integrated moving averages. On the other hand, deep learning and matrix factorization models have been recently proposed to tackle the same problem with more competitive performance. However, one major drawback of such models is that they tend to be overly complex in comparison to traditional techniques. In this paper, we report the results of prominent deep learning models with respect to a well-known machine learning baseline, a Gradient Boosting Regression Tree (GBRT) model. Similar to the deep neural network (DNN) models, we transform the time series forecasting task into a window-based regression problem. Furthermore, we feature-engineered the input and output structure of the GBRT model, such that, for each training window, the target values are concatenated with external features, and then flattened to form one input instance for a multi-output GBRT model. We conducted a comparative study on nine datasets for eight state-of-the-art deep-learning models that were presented at top-level conferences in the last years. The results demonstrate that the window-based input transformation boosts the performance of a simple GBRT model to levels that outperform all state-of-the-art DNN models evaluated in this paper.