# Fredformer: Frequency Debiased Transformer for Time Series Forecasting

arXiv: https://arxiv.org/abs/2406.09009 · HTML: https://mlsanity.com/paper/2406.09009/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Long-term Forecasting with TiDE: Time-series Dense Encoder](https://mlsanity.com/paper/2304.08424/) (2023)
- [Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting](https://mlsanity.com/paper/2103.07719/) (2021)
- [Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice](https://mlsanity.com/paper/2203.05962/) (2022)
- [Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences](https://mlsanity.com/paper/1610.09513/) (2016)
- [ContraNorm: A Contrastive Learning Perspective on Oversmoothing and Beyond](https://mlsanity.com/paper/2303.06562/) (2023)
- [A Memory-Network Based Solution for Multivariate Time-Series Forecasting](https://mlsanity.com/paper/1809.02105/) (2018)
- [Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling](https://mlsanity.com/paper/1911.06393/) (2019)
- [Short and Long-term Pattern Discovery Over Large-Scale Geo-Spatiotemporal Data](https://mlsanity.com/paper/1902.06792/) (2019)
- [Automated Sleep Staging via Parallel Frequency-Cut Attention](https://mlsanity.com/paper/2204.03173/) (2022)
- [Defining Fundamental Frequency for Almost Harmonic Signals](https://mlsanity.com/paper/2003.10767/) (2020)

## Built on by (most cited first, 30 of 45)

- [CATCH: Channel-Aware multivariate Time Series Anomaly Detection via Frequency Patching](https://mlsanity.com/paper/2410.12261/) (2024)
- [OLinear: A Linear Model for Time Series Forecasting in Orthogonally Transformed Domain](https://mlsanity.com/paper/2505.08550/) (2025)
- [Time-o1: Time-Series Forecasting Needs Transformed Label Alignment](https://mlsanity.com/paper/2505.17847/) (2025)
- [Tokenizing Single-Channel EEG with Time-Frequency Motif Learning](https://mlsanity.com/paper/2502.16060/) (2025)
- [DistDF: Time-Series Forecasting Needs Joint-Distribution Wasserstein Alignment](https://mlsanity.com/paper/2510.24574/) (2025)
- [TimeEmb: A Lightweight Static-Dynamic Disentanglement Framework for Time Series Forecasting](https://mlsanity.com/paper/2510.00461/) (2025)
- [Auto-Regressive Moving Diffusion Models for Time Series Forecasting](https://mlsanity.com/paper/2412.09328/) (2024)
- [Time Series Analysis in Frequency Domain: A Survey of Open Challenges, Opportunities and Benchmarks](https://mlsanity.com/paper/2504.07099/) (2025)
- [Quadratic Direct Forecast for Training Multi-Step Time-Series Forecast Models](https://mlsanity.com/paper/2511.00053/) (2025)
- [DeepFilter: A Transformer-style Framework for Accurate and Efficient Process Monitoring](https://mlsanity.com/paper/2501.01342/) (2025)
- [Unveiling the Potential of Text in High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/2501.07048/) (2025)
- [FredNormer: Frequency Domain Normalization for Non-stationary Time Series Forecasting](https://mlsanity.com/paper/2410.01860/) (2024)
- [LiNo: Advancing Recursive Residual Decomposition of Linear and Nonlinear Patterns for Robust Time Series Forecasting](https://mlsanity.com/paper/2410.17159/) (2024)
- [PHAT: Modeling Period Heterogeneity for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2602.00654/) (2026)
- [ST-Hyper: Learning High-Order Dependencies Across Multiple Spatial-Temporal Scales for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2509.02217/) (2025)
- [MillGNN: Learning Multi-Scale Lead-Lag Dependencies for Multi-Variate Time Series Forecasting](https://mlsanity.com/paper/2509.03852/) (2025)
- [TIFO: Time-Invariant Frequency Operator for Stationarity-Aware Representation Learning in Time Series](https://mlsanity.com/paper/2602.17122/) (2026)
- [Revisiting PCA for time series reduction in temporal dimension](https://mlsanity.com/paper/2412.19423/) (2024)
- [HADL Framework for Noise Resilient Long-Term Time Series Forecasting](https://mlsanity.com/paper/2502.10569/) (2025)
- [A Set-Sequence Model for Time Series](https://mlsanity.com/paper/2505.11243/) (2025)
- [Sonnet: Spectral Operator Neural Network for Multivariable Time Series Forecasting](https://mlsanity.com/paper/2505.15312/) (2025)
- [Mixture of Low Rank Adaptation with Partial Parameter Sharing for Time Series Forecasting](https://mlsanity.com/paper/2505.17872/) (2025)
- [Fremer: Lightweight and Effective Frequency Transformer for Workload Forecasting in Cloud Services](https://mlsanity.com/paper/2507.12908/) (2025)
- [Operator Flow Matching for Timeseries Forecasting](https://mlsanity.com/paper/2510.15101/) (2025)
- [FreIE: Low-Frequency Spectral Bias in Neural Networks for Time-Series Tasks](https://mlsanity.com/paper/2510.25800/) (2025)
- [BEAT: Balanced Frequency Adaptive Tuning for Long-Term Time-Series Forecasting](https://mlsanity.com/paper/2501.19065/) (2025)
- [OLMA: One Loss for More Accurate Time Series Forecasting](https://mlsanity.com/paper/2505.11567/) (2025)
- [The Power of Architecture: Deep Dive into Transformer Architectures for Long-Term Time Series Forecasting](https://mlsanity.com/paper/2507.13043/) (2025)
- [Filter then Attend: Improving attention-based Time Series Forecasting with Spectral Filtering](https://mlsanity.com/paper/2508.20206/) (2025)
- [Data-Efficient Psychiatric Disorder Detection via Self-supervised Learning on Frequency-enhanced Brain Networks](https://mlsanity.com/paper/2509.10524/) (2025)

## Abstract (arXiv)

The Transformer model has shown leading performance in time series forecasting. Nevertheless, in some complex scenarios, it tends to learn low-frequency features in the data and overlook high-frequency features, showing a frequency bias. This bias prevents the model from accurately capturing important high-frequency data features. In this paper, we undertook empirical analyses to understand this bias and discovered that frequency bias results from the model disproportionately focusing on frequency features with higher energy. Based on our analysis, we formulate this bias and propose Fredformer, a Transformer-based framework designed to mitigate frequency bias by learning features equally across different frequency bands. This approach prevents the model from overlooking lower amplitude features important for accurate forecasting. Extensive experiments show the effectiveness of our proposed approach, which can outperform other baselines in different real-world time-series datasets. Furthermore, we introduce a lightweight variant of the Fredformer with an attention matrix approximation, which achieves comparable performance but with much fewer parameters and lower computation costs. The code is available at: https://github.com/chenzRG/Fredformer