Data to 7 Oct 2026

Transformers in Time Series: A Survey

arXiv 2202.07125 ↗First posted 15 Feb 2022cs.LG cs.AI eess.SPcited by 489

Not on the map. Why?

Abstract (arXiv)

Transformers have achieved superior performances in many tasks in natural language processing and computer vision, which also triggered great interest in the time series community. Among multiple advantages of Transformers, the ability to capture long-range dependencies and interactions is especially attractive for time series modeling, leading to exciting progress in various time series applications. In this paper, we systematically review Transformer schemes for time series modeling by highlighting their strengths as well as limitations. In particular, we examine the development of time series Transformers in two perspectives. From the perspective of network structure, we summarize the adaptations and modifications that have been made to Transformers in order to accommodate the challenges in time series analysis. From the perspective of applications, we categorize time series Transformers based on common tasks including forecasting, anomaly detection, and classification. Empirically, we perform robust analysis, model size analysis, and seasonal-trend decomposition analysis to study how Transformers perform in time series. Finally, we discuss and suggest future directions to provide useful research guidance. To the best of our knowledge, this paper is the first work to comprehensively and systematically summarize the recent advances of Transformers for modeling time series data. We hope this survey will ignite further research interests in time series Transformers.

Key facts. Cited by 489 papers in the map (data through 7 Oct 2026). Builds on 53 of its 94 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

53 of 94 references.

References · 53

2017Attention Is All You Needcited by 59,932
2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
2020Language Models are Few-Shot Learnerscited by 32,199
2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
2013Auto-Encoding Variational Bayescited by 3,980
2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
2018Self-Attention with Relative Position Representationscited by 1,112
2021Transformers in Vision: A Surveycited by 1,009

38 more

2020Pre-Trained Image Processing Transformercited by 891
2020Efficient Transformers: A Surveycited by 784
2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
2020A Survey on Visual Transformercited by 653
2018Universal Transformerscited by 613
2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
2018Deep learning for time series classification: a reviewcited by 575
2017Convolutional Sequence to Sequence Learningcited by 550
2020Pre-trained Models for Natural Language Processing: A Surveycited by 504
2020Time Series Forecasting With Deep Learning: A Surveycited by 457
2020A Transformer-based Framework for Multivariate Time Series Representation Learningcited by 449
2020A Unifying Review of Deep and Shallow Anomaly Detectioncited by 423
2019Are Transformers universal approximators of sequence-to-sequence functions?cited by 404
2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 387
2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
2021Pre-Trained Models: Past, Present and Futurecited by 319
2020Transformer Hawkes Processcited by 230
2020Time Series Data Augmentation for Deep Learning: A Surveycited by 225
2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
2022TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Datacited by 211
2020Rethinking Positional Encoding in Language Pre-trainingcited by 197
2020A review on outlier/anomaly detection in time series datacited by 187
2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
2019The Evolved Transformercited by 180
2020Spatial-Temporal Transformer Networks for Traffic Flow Forecastingcited by 179
2020Lite Transformer with Long-Short Range Attentioncited by 127
2019An Attentive Survey of Attention Modelscited by 123
2020Deep Learning for Time Series Forecasting: Tutorial and Literature Surveycited by 117
2021Neural Temporal Point Processes: A Reviewcited by 113
2021Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoTcited by 104
2019Attention in Natural Language Processingcited by 100
2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
2021Transformer Embeddings of Irregularly Spaced Events and Their Participantscited by 90
2019Self-attention for raw optical Satellite Time Series Classificationcited by 75
2022Video Transformers: A Surveycited by 63
2021Gated Transformer Networks for Multivariate Time Series Classificationcited by 58
2018RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Seriescited by 54
2018Training Deeper Neural Machine Translation Models with Transparent Attentioncited by 37

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 23; 2023: 88; 2024: 138; 2025: 152; 2026: 88.

489 papers.

Sort:

By area:

2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
2023Large Language Models in Finance: A Surveycited by 223
2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139

38 more

2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
2024Large Language Models for Time Series: A Surveycited by 113
2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
2023DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Modelscited by 81
2022AirFormer: Predicting Nationwide Air Quality in China with Transformerscited by 73
2023OneNet: Enhancing Time Series Forecasting Models under Concept Drift by Online Ensemblingcited by 71
2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 71
2023Deep Learning for Time Series Classification and Extrinsic Regression: A Current Surveycited by 68
2023EasyTPP: Towards Open Benchmarking Temporal Point Processescited by 63
2025A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspectivecited by 58
2024Deep Learning for Multivariate Time Series Imputation: A Surveycited by 57
2023CARD: Channel Aligned Robust Blend Transformer for Time Series Forecastingcited by 55
2023SegRNN: Segment Recurrent Neural Network for Long-Term Time Series Forecastingcited by 53
2023A Survey on Time-Series Pre-Trained Modelscited by 52
2023MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecastingcited by 50
2024Position: What Can Large Language Models Tell Us about Time Series Analysiscited by 47
2024Continuum Attention for Neural Operatorscited by 39
2024From Similarity to Superiority: Channel Clustering for Time Series Forecastingcited by 37
2024xPatch: Dual-Stream Time Series Forecasting with Exponential Seasonal-Trend Decompositioncited by 35
2023Learning to Embed Time Series Patches Independentlycited by 34
2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 34
2024How Good is ChatGPT at Face Biometrics? A First Look into Recognition, Soft Biometrics, and Explainabilitycited by 33
2023Foundation Models for Weather and Climate Data Understanding: A Comprehensive Surveycited by 32
2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 32
2023Sumformer: Universal Approximation for Efficient Transformerscited by 30
2024Towards Neural Scaling Laws for Time Series Foundation Modelscited by 30
2023Mixture-of-Linear-Experts for Long-term Time Series Forecastingcited by 29
2024Unlocking the Power of LSTM for Long Term Time Series Forecastingcited by 29
2025Temporal Query Network for Efficient Multivariate Time Series Forecastingcited by 29
2023Toward a Foundation Model for Time Series Datacited by 28
2024Universal Time-Series Representation Learning: A Surveycited by 28
2024ConvTimeNet: A Deep Hierarchical Fully Convolutional Model for Multivariate Time Series Analysiscited by 26
2024Hierarchical Classification Auxiliary Network for Time Series Forecastingcited by 26

50 most cited of 489 All 489 (JSON)