mlsanityData to 7 Oct 2026

FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

arXiv 2201.12740 ↗First posted 30 Jan 2022cs.LG stat.MLcited by 1,200

Not on the map. Why?

Cited by 39 in State-space sequence models (S4, Mamba), but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for long-term prediction, we exploit the fact that most time series tend to have a sparse representation in well-known basis such as Fourier transform, and develop a frequency enhanced Transformer. Besides being more effective, the proposed method, termed as Frequency Enhanced Decomposed Transformer (bf FEDformer), is more efficient than standard Transformer with a linear complexity to the sequence length. Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by 14.8% and 22.6% for multivariate and univariate time series, respectively. Code is publicly available at https://github.com/MAZiqing/FEDformer.

Key facts. Cited by 1,200 papers in the map (data through 7 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 30 Jan 2022.

Builds on

32 of 57 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  8. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  10. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  11. 2020Reformer: The Efficient Transformercited by 1,658
  12. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
20 more
  1. 2020Rethinking Attention with Performerscited by 1,584
  2. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  3. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  4. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  5. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  6. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  7. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  8. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  9. 2021Global Filter Networks for Image Classificationcited by 257
  10. 2020Sparse Sinkhorn Attentioncited by 250
  11. 2017A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Predictioncited by 245
  12. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  13. 2019Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecastingcited by 161
  14. 2019Blockwise Self-Attention for Long Document Understandingcited by 158
  15. 2013Fast Training of Convolutional Networks through FFTscited by 89
  16. 2018RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Seriescited by 54
  17. 2020RobustTAD: Robust Time Series Anomaly Detection via Decomposition and Convolutional Neural Networkscited by 54
  18. 2021H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequencescited by 47
  19. 2019Sampled Softmax with Random Fourier Featurescited by 33
  20. 2020RobustPeriod: Time-Frequency Mining for Robust Multiple Periodicity Detectioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 31; 2023: 164; 2024: 325; 2025: 377; 2026: 303.

1,200 papers.

Sort:

By area:

  1. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
  2. 2022Are Transformers Effective for Time Series Forecasting?cited by 1,400
  3. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
  4. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  5. 2023Time-LLM: Time Series Forecasting by Reprogramming Large Language Modelscited by 680
  6. 2023A decoder-only foundation model for time-series forecastingcited by 650
  7. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 615
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  10. 2022Transformers in Time Series: A Surveycited by 489
  11. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  12. 2022Non-stationary Transformers: Exploring the Stationarity in Time Series Forecastingcited by 385
38 more
  1. 2023Long-term Forecasting with TiDE: Time-series Dense Encodercited by 327
  2. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 280
  3. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
  4. 2024TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variablescited by 241
  5. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  6. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  7. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  8. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 170
  9. 2023TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecastingcited by 170
  10. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  11. 2023TEST: Text Prototype Aligned Embedding to Activate LLM's Ability for Time Seriescited by 165
  12. 2024Are Language Models Actually Useful for Time Series Forecasting?cited by 163
  13. 2023TimeGPT-1cited by 149
  14. 2022FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecastingcited by 147
  15. 2023UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecastingcited by 139
  16. 2023TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecastingcited by 136
  17. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  18. 2023LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasterscited by 127
  19. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 126
  20. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 120
  21. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
  22. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
  23. 2024Is Mamba Effective for Time Series Forecasting?cited by 110
  24. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  25. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
  26. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 102
  27. 2023ForecastPFN: Synthetically-Trained Zero-Shot Forecastingcited by 102
  28. 2024Pathformer: Multi-scale Transformers with Adaptive Pathways for Time Series Forecastingcited by 102
  29. 2024CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patternscited by 102
  30. 2023The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecastingcited by 100
  31. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  32. 2024TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignmentcited by 97
  33. 2024UniTS: A Unified Multi-Task Time Series Modelcited by 93
  34. 2024From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflectioncited by 93
  35. 2024VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasterscited by 87
  36. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  37. 2023DiffSTG: Probabilistic Spatio-Temporal Graph Forecasting with Denoising Diffusion Modelscited by 81
  38. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time seriescited by 81

50 most cited of 1,200 All 1,200 (JSON)