# Time Series Forecasting (TSF) Using Various Deep Learning Models

arXiv: https://arxiv.org/abs/2204.11115 · HTML: https://mlsanity.com/paper/2204.11115/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 49 references that are in the map.
- First posted on arXiv 23 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN](https://mlsanity.com/paper/1803.04831/) (2018)
- [Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case](https://mlsanity.com/paper/2001.08317/) (2020)
- [Temporal Pattern Attention for Multivariate Time Series Forecasting](https://mlsanity.com/paper/1809.04206/) (2018)
- [Deep Air Quality Forecasting Using Hybrid Deep Learning Framework](https://mlsanity.com/paper/1812.04783/) (2018)
- [An Introductory Study on Time Series Modeling and Forecasting](https://mlsanity.com/paper/1302.6613/) (2013)
- [Quantifying the vanishing gradient and long distance dependency problem in recursive neural networks and recursive LSTMs](https://mlsanity.com/paper/1603.00423/) (2016)

## Built on by (most cited first, 4 of 4)

- [Deep Learning and Foundation Models for Weather Prediction: A Survey](https://mlsanity.com/paper/2501.06907/) (2025)
- [Explainable Parallel RCNN with Novel Feature Representation for Time Series Forecasting](https://mlsanity.com/paper/2305.04876/) (2023)
- [What-If Analysis of Large Language Models: Explore the Game World Using Proactive Thinking](https://mlsanity.com/paper/2509.04791/) (2025)
- [Predicting Grain Growth in Polycrystalline Materials Using Deep Learning Time Series Models](https://mlsanity.com/paper/2511.11630/) (2025)

## Abstract (arXiv)

Time Series Forecasting (TSF) is used to predict the target variables at a future time point based on the learning from previous time points. To keep the problem tractable, learning methods use data from a fixed length window in the past as an explicit input. In this paper, we study how the performance of predictive models change as a function of different look-back window sizes and different amounts of time to predict into the future. We also consider the performance of the recent attention-based Transformer models, which has had good success in the image processing and natural language processing domains. In all, we compare four different deep learning methods (RNN, LSTM, GRU, and Transformer) along with a baseline method. The dataset (hourly) we used is the Beijing Air Quality Dataset from the UCI website, which includes a multivariate time series of many factors measured on an hourly basis for a period of 5 years (2010-14). For each model, we also report on the relationship between the performance and the look-back window sizes and the number of predicted time points into the future. Our experiments suggest that Transformer models have the best performance with the lowest Mean Average Errors (MAE = 14.599, 23.273) and Root Mean Square Errors (RSME = 23.573, 38.131) for most of our single-step and multi-steps predictions. The best size for the look-back window to predict 1 hour into the future appears to be one day, while 2 or 4 days perform the best to predict 3 hours into the future.