# Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective

arXiv: https://arxiv.org/abs/2310.06379 · HTML: https://mlsanity.com/paper/2310.06379/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Exponential expressivity in deep neural networks through transient chaos](https://mlsanity.com/paper/1606.05340/) (2016)
- [Factorized Fourier Neural Operators](https://mlsanity.com/paper/2111.13802/) (2021)
- [Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers](https://mlsanity.com/paper/2111.13587/) (2021)
- [U-NO: U-shaped Neural Operators](https://mlsanity.com/paper/2204.11127/) (2022)
- [Deep Information Propagation](https://mlsanity.com/paper/1611.01232/) (2016)
- [Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks](https://mlsanity.com/paper/1806.05393/) (2018)
- [Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice](https://mlsanity.com/paper/1711.04735/) (2017)
- [Clifford Neural Layers for PDE Modeling](https://mlsanity.com/paper/2209.04934/) (2022)
- [Mean Field Residual Networks: On the Edge of Chaos](https://mlsanity.com/paper/1712.08969/) (2017)
- [Wavelet neural operator: a neural operator for parametric partial differential equations](https://mlsanity.com/paper/2205.02191/) (2022)
- [Lie Point Symmetry Data Augmentation for Neural PDE Solvers](https://mlsanity.com/paper/2202.07643/) (2022)
- [The Emergence of Spectral Universality in Deep Networks](https://mlsanity.com/paper/1802.09979/) (2018)
- [Seismic wave propagation and inversion with Neural Operators](https://mlsanity.com/paper/2108.05421/) (2021)
- [Group Equivariant Fourier Neural Operators for Partial Differential Equations](https://mlsanity.com/paper/2306.05697/) (2023)
- [Solving PDE-constrained Control Problems Using Operator Learning](https://mlsanity.com/paper/2111.04941/) (2021)
- [Transform Once: Efficient Operator Learning in Frequency Domain](https://mlsanity.com/paper/2211.14453/) (2022)
- [On the Selection of Initialization and Activation Function for Deep Neural Networks](https://mlsanity.com/paper/1805.08266/) (2018)
- [Multiphase flow prediction with deep neural networks](https://mlsanity.com/paper/1910.09657/) (2019)

## Built on by (most cited first, 11 of 11)

- [KANO: Kolmogorov-Arnold Neural Operator](https://mlsanity.com/paper/2509.16825/) (2025)
- [Diagnosing Failure Modes of Neural Operators Across Diverse PDE Families](https://mlsanity.com/paper/2601.11428/) (2026)
- [Spectral Convolutional Conditional Neural Processes](https://mlsanity.com/paper/2404.13182/) (2024)
- [Graph Fourier Neural ODEs: Modeling Spatial-temporal Multi-scales in Molecular Dynamics](https://mlsanity.com/paper/2411.01600/) (2024)
- [PDEInvBench: A Comprehensive Dataset and Design Space Exploration of Neural Networks for PDE Inverse Problems](https://mlsanity.com/paper/2605.25353/) (2026)
- [Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators](https://mlsanity.com/paper/2506.19396/) (2025)
- [Multi-Scale Wavelet Transformers for Operator Learning of Dynamical Systems](https://mlsanity.com/paper/2602.01486/) (2026)
- [Analysis of Fourier Neural Operators via Effective Field Theory](https://mlsanity.com/paper/2507.21833/) (2025)
- [PGOT: A Physics-Geometry Operator Transformer for Complex PDEs](https://mlsanity.com/paper/2512.23192/) (2025)
- [A neural operator view on U-Nets for inverse imaging problems](https://mlsanity.com/paper/2608.05839/) (2026)
- [Sensitivity Fidelity Reveals Physical Dependencies Beyond Forecast Accuracy in Deep-Learning Earth-System Models](https://mlsanity.com/paper/2610.15540/) (2026)

## Abstract (arXiv)

In this paper, we explores the expressivity and trainability of the Fourier Neural Operator (FNO). We establish a mean-field theory for the FNO, analyzing the behavior of the random FNO from an edge of chaos perspective. Our investigation into the expressivity of a random FNO involves examining the ordered-chaos phase transition of the network based on the weight distribution. This phase transition demonstrates characteristics unique to the FNO, induced by mode truncation, while also showcasing similarities to those of densely connected networks. Furthermore, we identify a connection between expressivity and trainability: the ordered and chaotic phases correspond to regions of vanishing and exploding gradients, respectively. This finding provides a practical prerequisite for the stable training of the FNO. Our experimental results corroborate our theoretical findings.