# An Experimental Review on Deep Learning Architectures for Time Series Forecasting

arXiv: https://arxiv.org/abs/2103.12057 · HTML: https://mlsanity.com/paper/2103.12057/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 106 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [Deep learning for time series classification: a review](https://mlsanity.com/paper/1809.04356/) (2018)
- [Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019](https://mlsanity.com/paper/1911.13288/) (2019)
- [Recurrent Neural Networks for Time Series Forecasting: Current Status and Future Directions](https://mlsanity.com/paper/1909.00590/) (2019)
- [Probabilistic Forecasting with Temporal Convolutional Neural Network](https://mlsanity.com/paper/1906.04397/) (2019)
- [Deep Learning for Time-Series Analysis](https://mlsanity.com/paper/1701.01887/) (2017)
- [Light Gated Recurrent Units for Speech Recognition](https://mlsanity.com/paper/1803.10225/) (2018)

## Built on by (most cited first, 30 of 47)

- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting](https://mlsanity.com/paper/2306.11025/) (2023)
- [xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories](https://mlsanity.com/paper/2410.16928/) (2024)
- [Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020-2022](https://mlsanity.com/paper/2305.04811/) (2023)
- [IoT Data Analytics in Dynamic Environments: From An Automated Machine Learning Perspective](https://mlsanity.com/paper/2209.08018/) (2022)
- [Well Googled is Half Done: Multimodal Forecasting of New Fashion Product Sales with Image-based Google Trends](https://mlsanity.com/paper/2109.09824/) (2021)
- [Federated Learning for 5G Base Station Traffic Forecasting](https://mlsanity.com/paper/2211.15220/) (2022)
- [Unsupervised Representation Learning for Time Series: A Review](https://mlsanity.com/paper/2308.01578/) (2023)
- [Deep Coupling Network For Multivariate Time Series Forecasting](https://mlsanity.com/paper/2402.15134/) (2024)
- [Time Series Analysis and Modeling to Forecast: a Survey](https://mlsanity.com/paper/2104.00164/) (2021)
- [A data filling methodology for time series based on CNN and (Bi)LSTM neural networks](https://mlsanity.com/paper/2204.09994/) (2022)
- [Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamics](https://mlsanity.com/paper/2508.18565/) (2025)
- [Comparing Prophet and Deep Learning to ARIMA in Forecasting Wholesale Food Prices](https://mlsanity.com/paper/2107.12770/) (2021)
- [A comparative study of non-deep learning, deep learning, and ensemble learning methods for sunspot number prediction](https://mlsanity.com/paper/2203.05757/) (2022)
- [Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals](https://mlsanity.com/paper/2203.05864/) (2022)
- [Bayesian Neural Networks for Macroeconomic Analysis](https://mlsanity.com/paper/2211.04752/) (2022)
- [A Distance Correlation-Based Approach to Characterize the Effectiveness of Recurrent Neural Networks for Time Series Forecasting](https://mlsanity.com/paper/2307.15830/) (2023)
- [Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting](https://mlsanity.com/paper/2407.19784/) (2024)
- [Multi-Spatio-temporal Fusion Graph Recurrent Network for Traffic forecasting](https://mlsanity.com/paper/2205.01480/) (2022)
- [Ensemble Modeling for Time Series Forecasting: an Adaptive Robust Optimization Approach](https://mlsanity.com/paper/2304.04308/) (2023)
- [TSPP: A Unified Benchmarking Tool for Time-series Forecasting](https://mlsanity.com/paper/2312.17100/) (2023)
- [Hypercomplex neural network in time series forecasting of stock data](https://mlsanity.com/paper/2401.04632/) (2024)
- [Mamba time series forecasting with uncertainty quantification](https://mlsanity.com/paper/2503.10873/) (2025)
- [Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks](https://mlsanity.com/paper/2505.20048/) (2025)
- [Neural MJD: Neural Non-Stationary Merton Jump Diffusion for Time Series Prediction](https://mlsanity.com/paper/2506.04542/) (2025)
- [Experimental study of time series forecasting methods for groundwater level prediction](https://mlsanity.com/paper/2209.13927/) (2022)
- [Temporal Saliency Detection Towards Explainable Transformer-based Timeseries Forecasting](https://mlsanity.com/paper/2212.07771/) (2022)
- [Rail Crack Propagation Forecasting Using Multi-horizons RNNs](https://mlsanity.com/paper/2309.01569/) (2023)
- [Unified Long-Term Time-Series Forecasting Benchmark](https://mlsanity.com/paper/2309.15946/) (2023)
- [A Study of Quantisation-aware Training on Time Series Transformer Models for Resource-constrained FPGAs](https://mlsanity.com/paper/2310.02654/) (2023)

## Abstract (arXiv)

In recent years, deep learning techniques have outperformed traditional models in many machine learning tasks. Deep neural networks have successfully been applied to address time series forecasting problems, which is a very important topic in data mining. They have proved to be an effective solution given their capacity to automatically learn the temporal dependencies present in time series. However, selecting the most convenient type of deep neural network and its parametrization is a complex task that requires considerable expertise. Therefore, there is a need for deeper studies on the suitability of all existing architectures for different forecasting tasks. In this work, we face two main challenges: a comprehensive review of the latest works using deep learning for time series forecasting; and an experimental study comparing the performance of the most popular architectures. The comparison involves a thorough analysis of seven types of deep learning models in terms of accuracy and efficiency. We evaluate the rankings and distribution of results obtained with the proposed models under many different architecture configurations and training hyperparameters. The datasets used comprise more than 50000 time series divided into 12 different forecasting problems. By training more than 38000 models on these data, we provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives, with LSTMs obtaining the most accurate forecasts. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.