# FreEformer: Frequency Enhanced Transformer for Multivariate Time Series Forecasting

arXiv: https://arxiv.org/abs/2501.13989 · HTML: https://mlsanity.com/paper/2501.13989/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 46 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting](https://mlsanity.com/paper/2205.14415/) (2022)
- [TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting](https://mlsanity.com/paper/2405.14616/) (2024)
- [FITS: Modeling Time Series with 10k Parameters](https://mlsanity.com/paper/2307.03756/) (2023)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [FLatten Transformer: Vision Transformer using Focused Linear Attention](https://mlsanity.com/paper/2308.00442/) (2023)
- [FiLM: Frequency improved Legendre Memory Model for Long-term Time Series Forecasting](https://mlsanity.com/paper/2205.08897/) (2022)
- [FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective](https://mlsanity.com/paper/2311.06190/) (2023)
- [The Capacity and Robustness Trade-off: Revisiting the Channel Independent Strategy for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2304.05206/) (2023)
- [Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2307.11494/) (2023)
- [FreDF: Learning to Forecast in the Frequency Domain](https://mlsanity.com/paper/2402.02399/) (2024)
- [Flowformer: Linearizing Transformers with Conservation Flows](https://mlsanity.com/paper/2202.06258/) (2022)

## Built on by (most cited first, 18 of 18)

- [DBLoss: Decomposition-based Loss Function for Time Series Forecasting](https://mlsanity.com/paper/2510.23672/) (2025)
- [Enhancing Time Series Forecasting through Selective Representation Spaces: A Patch Perspective](https://mlsanity.com/paper/2510.14510/) (2025)
- [OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain](https://mlsanity.com/paper/2505.08550/) (2025)
- [Time-o1: Time-Series Forecasting Needs Transformed Label Alignment](https://mlsanity.com/paper/2505.17847/) (2025)
- [Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarks](https://mlsanity.com/paper/2504.07099/) (2025)
- [FRWKV:Frequency-Domain Linear Attention for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2512.07539/) (2025)
- [vLinear: A Powerful Linear Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2601.13768/) (2026)
- [A Physics Prior-Guided Dual-Stream Attention Network for Motion Prediction of Elastic Bragg Breakwaters](https://mlsanity.com/paper/2510.14250/) (2025)
- [Airport Passenger Flow Forecasting via Deformable Temporal-Spectral Transformer Approach](https://mlsanity.com/paper/2512.11845/) (2025)
- [Dualformer: Time-Frequency Dual Domain Learning for Long-term Time Series Forecasting](https://mlsanity.com/paper/2601.15669/) (2026)
- [FAIR-ESI: Feature Adaptive Importance Refinement for Electrophysiological Source Imaging](https://mlsanity.com/paper/2601.15731/) (2026)
- [Deep Time-Series Forecasting in 10 Years: A Survey](https://mlsanity.com/paper/2603.19899/) (2026)
- [FAiT: Frequency-Aware Inverted Transformer for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2606.01306/) (2026)
- [VLBM: Variational Latent Basis Modeling for OOD Robust Multivariate Time Series Forecasting](https://mlsanity.com/paper/2606.02138/) (2026)
- [Beyond Uniform Tokens: Adaptive Compression for Time Series Language Models](https://mlsanity.com/paper/2606.13624/) (2026)
- [TA-SparseMG: Trend-Aware Sparse Forecasting via Multi-Scale Gating for Long-Term Time Series](https://mlsanity.com/paper/2606.27908/) (2026)
- [Frequency-Domain Multi-Modality Transportation Modeling](https://mlsanity.com/paper/2607.08475/) (2026)
- [Revitalizing Medical Time Series with Vision-Informed Retrieval: A Vision-Language Perspective](https://mlsanity.com/paper/2609.34652/) (2026)

## Abstract (arXiv)

This paper presents FreEformer, a simple yet effective model that leverages a Frequency Enhanced Transformer for multivariate time series forecasting. Our work is based on the assumption that the frequency spectrum provides a global perspective on the composition of series across various frequencies and is highly suitable for robust representation learning. Specifically, we first convert time series into the complex frequency domain using the Discrete Fourier Transform (DFT). The Transformer architecture is then applied to the frequency spectra to capture cross-variate dependencies, with the real and imaginary parts processed independently. However, we observe that the vanilla attention matrix exhibits a low-rank characteristic, thus limiting representation diversity. This could be attributed to the inherent sparsity of the frequency domain and the strong-value-focused nature of Softmax in vanilla attention. To address this, we enhance the vanilla attention mechanism by introducing an additional learnable matrix to the original attention matrix, followed by row-wise L1 normalization. Theoretical analysis~demonstrates that this enhanced attention mechanism improves both feature diversity and gradient flow. Extensive experiments demonstrate that FreEformer consistently outperforms state-of-the-art models on eighteen real-world benchmarks covering electricity, traffic, weather, healthcare and finance. Notably, the enhanced attention mechanism also consistently improves the performance of state-of-the-art Transformer-based forecasters.