mlsanityData to 7 Oct 2026

Are Transformers Effective for Time Series Forecasting?

arXiv 2205.13504 ↗First posted 26 May 2022cs.AI cs.LGcited by 1,400

Not on the map. Why?

Cited by 35 in State-space sequence models (S4, Mamba), but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, there has been a surge of Transformer-based solutions for the long-term time series forecasting (LTSF) task. Despite the growing performance over the past few years, we question the validity of this line of research in this work. Specifically, Transformers is arguably the most successful solution to extract the semantic correlations among the elements in a long sequence. However, in time series modeling, we are to extract the temporal relations in an ordered set of continuous points. While employing positional encoding and using tokens to embed sub-series in Transformers facilitate preserving some ordering information, the nature of the permutation-invariant self-attention mechanism inevitably results in temporal information loss. To validate our claim, we introduce a set of embarrassingly simple one-layer linear models named LTSF-Linear for comparison. Experimental results on nine real-life datasets show that LTSF-Linear surprisingly outperforms existing sophisticated Transformer-based LTSF models in all cases, and often by a large margin. Moreover, we conduct comprehensive empirical studies to explore the impacts of various design elements of LTSF models on their temporal relation extraction capability. We hope this surprising finding opens up new research directions for the LTSF task. We also advocate revisiting the validity of Transformer-based solutions for other time series analysis tasks (e.g., anomaly detection) in the future. Code is available at: urlhttps://github.com/cure-lab/LTSF-Linear.

Key facts. Cited by 1,400 papers in the map (data through 7 Oct 2026). Builds on 20 of its 36 references that are in the map. First posted on arXiv 26 May 2022.

Builds on

20 of 36 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  6. 2020Reformer: The Efficient Transformercited by 1,658
  7. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  8. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  9. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  10. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  11. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  12. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
8 more
  1. 2022Transformers in Time Series: A Surveycited by 489
  2. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 335
  3. 2021Multimodal Motion Prediction with Stacked Transformerscited by 207
  4. 2020An Evaluation of Change Point Detection Algorithmscited by 62
  5. 2022Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Versioncited by 59
  6. 2021Do We Really Need Deep Learning Models for Time Series Forecasting?cited by 45
  7. 2022DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimationcited by 29
  8. 2019Recurrent Neural Networks for Time Series Forecastingcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 142; 2024: 375; 2025: 454; 2026: 418.

1,400 papers.

Sort:

By area:

  1. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  2. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  3. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  4. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  5. 2023A decoder-only foundation model for time-series forecastingcited by 650
  6. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  7. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  8. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  9. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
  10. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  11. 2024Titans: Learning to Memorize at Test Timecited by 303
  12. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
38 more
  1. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
  2. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  3. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 198
  4. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  5. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  6. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  7. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  8. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  9. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  10. 2023TimeGPT-1cited by 149
  11. 2023Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Surveycited by 140
  12. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
  13. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  14. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  15. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  16. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
  17. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
  18. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 110
  19. 2024Is Mamba Effective for Time Series Forecasting?cited by 110
  20. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  21. 2025TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learningcited by 107
  22. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  23. 2024DUET: Dual Clustering Enhanced Multivariate Time Series Forecastingcited by 104
  24. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
  25. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  26. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  27. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  28. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  29. 2024CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patchingcited by 91
  30. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  31. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  32. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 81
  33. 2023Non-autoregressive Conditional Diffusion Models for Time Series Predictioncited by 77
  34. 2024FreDF: Learning to Forecast in the Frequency Domaincited by 77
  35. 2024SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusioncited by 75
  36. 2023OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensemblingcited by 71
  37. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
  38. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 65

50 most cited of 1,400 All 1,400 (JSON)