mlsanityData to 7 Oct 2026

FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting

arXiv 2205.08897 ↗First posted 18 May 2022cs.LG stat.MLcited by 147

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that deep learning models such as RNNs and Transformers have brought significant performance gains for long-term forecasting of time series because they effectively utilize historical information. We found, however, that there is still great room for improvement in how to preserve historical information in neural networks while avoiding overfitting to noise presented in the history. Addressing this allows better utilization of the capabilities of deep learning models. To this end, we design a Frequency improved Legendre Memory model, or bf FiLM: it applies Legendre Polynomials projections to approximate historical information, uses Fourier projection to remove noise, and adds a low-rank approximation to speed up computation. Our empirical studies show that the proposed FiLM significantly improves the accuracy of state-of-the-art models in multivariate and univariate long-term forecasting by (20.3%, 22.6%), respectively. We also demonstrate that the representation module developed in this work can be used as a general plug-in to improve the long-term prediction performance of other deep learning modules. Code is available at https://github.com/tianzhou2011/FiLM/

Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 18 May 2022.

Builds on

35 of 61 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  9. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,141
  10. 2020Reformer: The Efficient Transformercited by 1,658
  11. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,654
  12. 2020Rethinking Attention with Performerscited by 1,584
23 more
  1. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  2. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,200
  3. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  4. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 869
  5. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  6. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  7. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  8. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 606
  9. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  10. 2022Transformers in Time Series: A Surveycited by 489
  11. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  12. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  13. 2021Global Filter Networks for Image Classificationcited by 257
  14. 2020Sparse Sinkhorn Attentioncited by 250
  15. 2017A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Predictioncited by 245
  16. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 229
  17. 2020Time Series Data Augmentation for Deep Learning: A Surveycited by 225
  18. 2019Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecastingcited by 161
  19. 2019Blockwise Self-Attention for Long Document Understandingcited by 158
  20. 2022N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingcited by 146
  21. 2018Multilevel Wavelet Decomposition Network for Interpretable Time Series Analysiscited by 61
  22. 2021H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequencescited by 47
  23. 2018Learning Long Term Dependencies via Fourier Recurrent Unitscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 36; 2024: 39; 2025: 37; 2026: 31.

147 papers.

Sort:

By area:

  1. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  2. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
  3. 2024TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methodscited by 242
  4. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  5. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  6. 2023FITS: Modeling Time Series with 10k Parameterscited by 185
  7. 2023Frequency-domain MLPs are More Effective Learners in Time Series Forecastingcited by 170
  8. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 126
  9. 2023Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysiscited by 113
  10. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 113
  11. 2024CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuningcited by 105
  12. 2024SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameterscited by 103
38 more
  1. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  2. 2023Non-autoregressive Conditional Diffusion Models for Time Series Predictioncited by 77
  3. 2024AutoTimes: Autoregressive Time Series Forecasters via Large Language Modelscited by 63
  4. 2023CARD: Channel Aligned Robust Blend Transformer for Time Series Forecastingcited by 55
  5. 2022Scaleformer: Iterative Multi-scale Refining Transformers for Time Series Forecastingcited by 54
  6. 2023Effectively Modeling Time Series with Simple Discrete State Spacescited by 54
  7. 2023A Survey on Time-Series Pre-Trained Modelscited by 52
  8. 2023DSformer: A Double Sampling Transformer for Multivariate Time Series Long-term Predictioncited by 49
  9. 2024Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modelingcited by 44
  10. 2024FilterNet: Harnessing Frequency Filters for Time Series Forecastingcited by 44
  11. 2023On Sparse Modern Hopfield Modelcited by 41
  12. 2023BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basiscited by 41
  13. 2023A Survey on Deep Learning based Time Series Analysis with Frequency Transformationcited by 40
  14. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  15. 2024From Similarity to Superiority: Channel Clustering for Time Series Forecastingcited by 37
  16. 2025Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrativecited by 35
  17. 2024TSI-Bench: Benchmarking Time Series Imputationcited by 33
  18. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 32
  19. 2025TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecastingcited by 27
  20. 2023PETformer: Long-term Time Series Forecasting via Placeholder-enhanced Transformercited by 22
  21. 2023Frequency-Aware Masked Autoencoders for Multimodal Pretraining on Biosignalscited by 22
  22. 2024BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Modelcited by 21
  23. 2022First De-Trend then Attend: Rethinking Attention for Time-Series Forecastingcited by 19
  24. 2023Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Meancited by 18
  25. 2024Are Self-Attentions Effective for Time Series Forecasting?cited by 18
  26. 2024TimeRAG: BOOSTING LLM Time Series Forecasting via Retrieval-Augmented Generationcited by 18
  27. 2025FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecastingcited by 18
  28. 2024Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspectivecited by 17
  29. 2024Intervention-Aware Forecasting: Breaking Historical Limits from a System Perspectivecited by 16
  30. 2022FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecastingcited by 15
  31. 2023Improving day-ahead Solar Irradiance Time Series Forecasting by Leveraging Spatio-Temporal Contextcited by 13
  32. 2023Revisiting Long-term Time Series Forecasting: An Investigation on Linear Mappingcited by 11
  33. 2025Multi-period Learning for Financial Time Series Forecastingcited by 11
  34. 2023OneShotSTL: One-Shot Seasonal-Trend Decomposition For Online Time Series Anomaly Detection And Forecastingcited by 10
  35. 2024DAM: Towards A Foundation Model for Time Series Forecastingcited by 10
  36. 2023Towards Expressive Spectral-Temporal Graph Neural Networks for Time Series Forecastingcited by 8
  37. 2024Self-Supervised Contrastive Learning for Long-term Forecastingcited by 8
  38. 2024ATFNet: Adaptive Time-Frequency Ensembled Network for Long-term Time Series Forecastingcited by 8

50 most cited of 147 All 147 (JSON)