# FAN: Fourier Analysis Networks

arXiv: https://arxiv.org/abs/2410.02675 · HTML: https://mlsanity.com/paper/2410.02675/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 69 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting](https://mlsanity.com/paper/2106.13008/) (2021)
- [FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting](https://mlsanity.com/paper/2201.12740/) (2022)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [FNet: Mixing Tokens with Fourier Transforms](https://mlsanity.com/paper/2105.03824/) (2021)
- [Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models](https://mlsanity.com/paper/2402.15938/) (2024)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [A Survey of Complex-Valued Neural Networks](https://mlsanity.com/paper/2101.12249/) (2021)
- [Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)](https://mlsanity.com/paper/2205.00664/) (2022)
- [Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization](https://mlsanity.com/paper/2204.10533/) (2022)
- [Fourier Neural Networks: A Comparative Study](https://mlsanity.com/paper/1902.03011/) (2019)
- [Efficient Twitter Sentiment Classification using Subjective Distant Supervision](https://mlsanity.com/paper/1701.03051/) (2017)
- [Conditional Generation of Periodic Signals with Fourier-Based Decoder](https://mlsanity.com/paper/2110.12365/) (2021)
- [Periodic Extrapolative Generalisation in Neural Networks](https://mlsanity.com/paper/2209.10280/) (2022)
- [Oscillatory Fourier Neural Network: A Compact and Efficient Architecture for Sequential Processing](https://mlsanity.com/paper/2109.13090/) (2021)

## Built on by (most cited first, 13 of 13)

- [TabKAN: Advancing Tabular Data Analysis using Kolmogorov-Arnold Network](https://mlsanity.com/paper/2504.06559/) (2025)
- [Kolmogorov-Arnold Attention: Is Learnable Attention Better For Vision Transformers?](https://mlsanity.com/paper/2503.10632/) (2025)
- [A novel Fourier Adjacency Transformer for advanced EEG emotion recognition](https://mlsanity.com/paper/2503.13465/) (2025)
- [BrainCast: A Spatio-Temporal Forecasting Model for Whole-Brain fMRI Time Series Prediction](https://mlsanity.com/paper/2603.13361/) (2026)
- [FAN-Unet: Enhancing Unet with vision Fourier Analysis Block for Biomedical Image Segmentation](https://mlsanity.com/paper/2411.18975/) (2024)
- [Convolutional Fourier Analysis Network (CFAN): A Unified Time-Frequency Approach for ECG Classification](https://mlsanity.com/paper/2502.00497/) (2025)
- [AR-KAN: Autoregressive-Weight-Enhanced Kolmogorov-Arnold Network for Time Series Forecasting](https://mlsanity.com/paper/2509.02967/) (2025)
- [MCIGLE: Multimodal Exemplar-Free Class-Incremental Graph Learning](https://mlsanity.com/paper/2509.06219/) (2025)
- [The Language of Touch: Translating Vibrations into Text with Dual-Branch Learning](https://mlsanity.com/paper/2603.26804/) (2026)
- [Spectral Transformer Neural Processes](https://mlsanity.com/paper/2605.09498/) (2026)
- [Frequency Decoupled Framework for Screen Content Image Super-Resolution](https://mlsanity.com/paper/2606.09029/) (2026)
- [FM-LLM: A frequency-enhanced mixture-of-experts framework for adapting LLMs to time series forecasting](https://mlsanity.com/paper/2608.11623/) (2026)
- [Digital Self-Interference Cancellation for In-Band Full-Duplex Systems Based on Transformer Neural Operators](https://mlsanity.com/paper/2610.19490/) (2026)

## Abstract (arXiv)

Despite the remarkable successes of general-purpose neural networks, such as MLPs and Transformers, we find that they exhibit notable shortcomings in modeling and reasoning about periodic phenomena, achieving only marginal performance within the training domain and failing to generalize effectively to out-of-domain (OOD) scenarios. Periodicity is ubiquitous throughout nature and science. Therefore, neural networks should be equipped with the essential ability to model and handle periodicity. In this work, we propose FAN, a novel neural network that effectively addresses periodicity modeling challenges while offering broad applicability similar to MLP with fewer parameters and FLOPs. Periodicity is naturally integrated into FAN's structure and computational processes by introducing the Fourier Principle. Unlike existing Fourier-based networks, which possess particular periodicity modeling abilities but face challenges in scaling to deeper networks and are typically designed for specific tasks, our approach overcomes this challenge to enable scaling to large-scale models and maintains general-purpose modeling capability. Through extensive experiments, we demonstrate the superiority of FAN in periodicity modeling tasks and the effectiveness and generalizability of FAN across a range of real-world tasks. Moreover, we reveal that compared to existing Fourier-based networks, FAN accommodates both periodicity modeling and general-purpose modeling well.