mlsanityData to 14 Oct 2026

Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting

arXiv 2012.07436 ↗First posted 14 Dec 2020cs.LG cs.AI cs.IRcited by 2,169

Not on the map. Why?

Cited by 56 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.

Key facts. Cited by 2,169 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 70; 2022: 159; 2023: 281; 2024: 480; 2025: 628; 2026: 551.

2,169 papers.

Sort:

By area:

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  2. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  3. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  4. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,424
  5. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  6. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  7. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  8. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 700
  9. 2023A decoder-only foundation model for time-series forecastingcited by 692
  10. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  11. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  12. 2022Transformers in Time Series: A Surveycited by 497
38 more
  1. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  2. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 471
  3. 2021A Survey of Transformerscited by 401
  4. 2021Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancycited by 394
  5. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 391
  6. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 346
  7. 2021SCINet: Time Series Modeling and Forecasting with Sample Convolution and Interactioncited by 338
  8. 2023ClimaX: A foundation model for weather and climatecited by 337
  9. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 332
  10. 2024Titans: Learning to Memorize at Test Timecited by 328
  11. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  12. 2021Monash Time Series Forecasting Archivecited by 267
  13. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 249
  14. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 240
  15. 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 239
  16. 2024GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluationcited by 212
  17. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
  18. 2022Diffusion-based Time Series Imputation and Forecasting with Structured State Space Modelscited by 198
  19. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 197
  20. 2023FITS: Modeling Time Series with 10k Parameterscited by 195
  21. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 174
  22. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  23. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 172
  24. 2022SAITS: Self-Attention-based Imputation for Time Seriescited by 169
  25. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 169
  26. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 168
  27. 2022Less Is More: Fast Multivariate Time Series Forecasting with Light Sampling-oriented MLP Structurescited by 160
  28. 2023TimeGPT-1cited by 158
  29. 2023Towards Better Dynamic Graph Learning: New Architecture and Unified Librarycited by 157
  30. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 148
  31. 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
  32. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  33. 2022CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecastingcited by 144
  34. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 144
  35. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 139
  36. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  37. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 129
  38. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 127

50 most cited of 2,169 All 2,169 (JSON)