# Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

arXiv: https://arxiv.org/abs/2001.08317 · HTML: https://mlsanity.com/paper/2001.08317/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2020.

## Built on by (most cited first, 30 of 129)

- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2310.08278/) (2023)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecasting](https://mlsanity.com/paper/2206.04038/) (2022)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [Representation learning for neural population activity with Neural Data Transformers](https://mlsanity.com/paper/2108.01210/) (2021)
- [PanGu-Coder: Program Synthesis with Function-Level Language Modeling](https://mlsanity.com/paper/2207.11280/) (2022)
- [Transformers in Time-series Analysis: A Tutorial](https://mlsanity.com/paper/2205.01138/) (2022)
- [Annotating Columns with Pre-trained Language Models](https://mlsanity.com/paper/2104.01785/) (2021)
- [BuildingsBench: A Large-Scale Dataset of 900K Buildings and Benchmark for Short-Term Load Forecasting](https://mlsanity.com/paper/2307.00142/) (2023)
- [Preformer: Predictive Transformer with Multi-Scale Segment-wise Correlations for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2202.11356/) (2022)
- [Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction](https://mlsanity.com/paper/2106.15842/) (2021)
- [Transformer-Based Self-Supervised Learning for Emotion Recognition](https://mlsanity.com/paper/2204.05103/) (2022)
- [Epicasting: An Ensemble Wavelet Neural Network (EWNet) for Forecasting Epidemics](https://mlsanity.com/paper/2206.10696/) (2022)
- [Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities](https://mlsanity.com/paper/2210.15359/) (2022)
- [Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset](https://mlsanity.com/paper/2407.02751/) (2024)
- [Automated Essay Scoring Using Transformer Models](https://mlsanity.com/paper/2110.06874/) (2021)
- [Online Metro Origin-Destination Prediction via Heterogeneous Information Aggregation](https://mlsanity.com/paper/2107.00946/) (2021)
- [MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series](https://mlsanity.com/paper/2205.02100/) (2022)
- [W-Transformers : A Wavelet-based Transformer Framework for Univariate Time Series Forecasting](https://mlsanity.com/paper/2209.03945/) (2022)
- [Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgery](https://mlsanity.com/paper/2212.01683/) (2022)
- [Diffusion Models for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2507.14507/) (2025)
- [Estimating the electrical power output of industrial devices with end-to-end time-series classification in the presence of label noise](https://mlsanity.com/paper/2105.00349/) (2021)
- [Time-series Transformer Generative Adversarial Networks](https://mlsanity.com/paper/2205.11164/) (2022)
- [A Systematic Review for Transformer-based Long-term Series Forecasting](https://mlsanity.com/paper/2310.20218/) (2023)
- [Deep Learning for Time Series Forecasting: A Survey](https://mlsanity.com/paper/2503.10198/) (2025)
- [Deep Autoregressive Models with Spectral Attention](https://mlsanity.com/paper/2107.05984/) (2021)
- [MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data](https://mlsanity.com/paper/2110.14354/) (2021)
- [From Pixels to Predictions: Spectrogram and Vision Transformer for Better Time Series Forecasting](https://mlsanity.com/paper/2403.11047/) (2024)

## Abstract (arXiv)

In this paper, we present a new approach to time series forecasting. Time series data are prevalent in many scientific and engineering disciplines. Time series forecasting is a crucial task in modeling time series data, and is an important area of machine learning. In this work we developed a novel method that employs Transformer-based machine learning models to forecast time series data. This approach works by leveraging self-attention mechanisms to learn complex patterns and dynamics from time series data. Moreover, it is a generic framework and can be applied to univariate and multivariate time series data, as well as time series embeddings. Using influenza-like illness (ILI) forecasting as a case study, we show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.