mlsanityData to 14 Oct 2026

iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

arXiv 2310.06625 ↗First posted 10 Oct 2023cs.LGcited by 931

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed events and distinct physical measurements, which may fail in learning variate-centric representations and result in meaningless attention maps. In this work, we reflect on the competent duties of Transformer components and repurpose the Transformer architecture without any modification to the basic components. We propose iTransformer that simply applies the attention and feed-forward network on the inverted dimensions. Specifically, the time points of individual series are embedded into variate tokens which are utilized by the attention mechanism to capture multivariate correlations; meanwhile, the feed-forward network is applied for each variate token to learn nonlinear representations. The iTransformer model achieves state-of-the-art on challenging real-world datasets, which further empowers the Transformer family with promoted performance, generalization ability across different variates, and better utilization of arbitrary lookback windows, making it a nice alternative as the fundamental backbone of time series forecasting. Code is available at this repository: https://github.com/thuml/iTransformer.

Key facts. Cited by 931 papers in the map (data through 14 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

28 of 38 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2016Layer Normalizationcited by 5,669
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  10. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  11. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  12. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
16 more
  1. 2020Reformer: The Efficient Transformercited by 1,668
  2. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  3. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  4. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  5. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  6. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  7. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
  8. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  9. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 753
  10. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 338
  11. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  12. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 139
  13. 2023SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modelingcited by 115
  14. 2023Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictorscited by 113
  15. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 101
  16. 2022Flowformer: Linearizing Transformers with Conservation Flowscited by 69

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 205; 2025: 341; 2026: 381.

931 papers.

Sort:

By area:

  1. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 614
  2. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  3. 2024Titans: Learning to Memorize at Test Timecited by 328
  4. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  5. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 248
  6. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  7. 2024Timer: Generative Pre-trained Transformers Are Large Time Series Modelscited by 208
  8. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 197
  9. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  10. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 168
  11. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
  12. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 116
38 more
  1. 2024Is Mamba Effective for Time Series Forecasting?cited by 114
  2. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 108
  3. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 106
  4. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  5. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 94
  6. 2024CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patchingcited by 94
  7. 2024AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoMLcited by 87
  8. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 87
  9. 2024Spatial-Temporal Large Language Model for Traffic Predictioncited by 83
  10. 2024Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusioncited by 81
  11. 2024FreDF: Learning to Forecast in the Frequency Domaincited by 79
  12. 2024SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusioncited by 76
  13. 2024TSLANet: Rethinking Transformers for Time Series Representation Learningcited by 74
  14. 2024Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysiscited by 74
  15. 2025This Time is Different: An Observability Perspective on Time Series Foundation Modelscited by 68
  16. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 65
  17. 2025A Comprehensive Survey of Deep Learning for Multivariate Time Series Forecasting: A Channel Strategy Perspectivecited by 58
  18. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  19. 2024Toto: Time Series Optimized Transformer for Observabilitycited by 48
  20. 2024FilterNet: Harnessing Frequency Filters for Time Series Forecastingcited by 47
  21. 2024Retrieval Augmented Time Series Forecastingcited by 46
  22. 2025DBLoss: Decomposition-based Loss Function for Time Series Forecastingcited by 45
  23. 2024SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attentioncited by 44
  24. 2024Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modelingcited by 44
  25. 2025R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimizationcited by 43
  26. 2024TimeBridge: Non-Stationarity Matters for Long-term Time Series Forecastingcited by 40
  27. 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 40
  28. 2024TOTEM: TOkenized Time Series EMbeddings for General Time Series Analysiscited by 39
  29. 2024Retrieval-Augmented Diffusion Models for Time Series Forecastingcited by 39
  30. 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 39
  31. 2025TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loopcited by 39
  32. 2025Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspectivecited by 39
  33. 2024From Similarity to Superiority: Channel Clustering for Time Series Forecastingcited by 37
  34. 2025ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Datasetcited by 37
  35. 2025Patch-wise Structural Loss for Time Series Forecastingcited by 35
  36. 2025Time Series Forecasting via Reasoning: A Slow-Thinking Approach with Reinforcement Fine-Tuned LLMscited by 34
  37. 2024Scaling Law for Time Series Forecastingcited by 33
  38. 2024TSI-Bench: Benchmarking Time Series Imputationcited by 33

50 most cited of 931 All 931 (JSON)