# Nonlocality and Nonlinearity Implies Universality in Operator Learning

arXiv: https://arxiv.org/abs/2304.13221 · HTML: https://mlsanity.com/paper/2304.13221/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [Fourier Neural Operator with Learned Deformations for PDEs on General Geometries](https://mlsanity.com/paper/2207.05209/) (2022)
- [Multipole Graph Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2006.09535/) (2020)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Model Reduction and Neural Networks for Parametric PDEs](https://mlsanity.com/paper/2005.03180/) (2020)
- [Multiwavelet-based Operator Learning for Differential Equations](https://mlsanity.com/paper/2109.13459/) (2021)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [NOMAD: Nonlinear Manifold Decoders for Operator Learning](https://mlsanity.com/paper/2206.03551/) (2022)
- [Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modeling](https://mlsanity.com/paper/2203.08205/) (2022)
- [A physics-informed operator regression framework for extracting data-driven continuum models](https://mlsanity.com/paper/2009.11992/) (2020)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [LNO: Laplace Neural Operator for Solving Differential Equations](https://mlsanity.com/paper/2303.10528/) (2023)
- [Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations](https://mlsanity.com/paper/2112.08125/) (2021)
- [Learning Dissipative Dynamics in Chaotic Systems](https://mlsanity.com/paper/2106.06898/) (2021)
- [Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuities](https://mlsanity.com/paper/2210.01074/) (2022)
- [Neural and spectral operator surrogates: unified construction and expression rate bounds](https://mlsanity.com/paper/2207.04950/) (2022)
- [Variationally Mimetic Operator Networks](https://mlsanity.com/paper/2209.12871/) (2022)
- [Nonlinear integro-differential operator regression with neural networks](https://mlsanity.com/paper/1810.08552/) (2018)

## Built on by (most cited first, 30 of 43)

- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Continuum Attention for Neural Operators](https://mlsanity.com/paper/2406.06486/) (2024)
- [Plasma Surrogate Modelling using Fourier Neural Operators](https://mlsanity.com/paper/2311.05967/) (2023)
- [Deep Operator Network Approximation Rates for Lipschitz Operators](https://mlsanity.com/paper/2307.09835/) (2023)
- [Active Learning for Neural PDE Solvers](https://mlsanity.com/paper/2408.01536/) (2024)
- [Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2403.08220/) (2024)
- [The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures](https://mlsanity.com/paper/2410.23440/) (2024)
- [LazyDINO: Fast, scalable, and efficiently amortized Bayesian inversion via structure-exploiting and surrogate-driven measure transport](https://mlsanity.com/paper/2411.12726/) (2024)
- [KANO: Kolmogorov-Arnold Neural Operator](https://mlsanity.com/paper/2509.16825/) (2025)
- [Performance of Neural and Polynomial Operator Surrogates](https://mlsanity.com/paper/2604.00689/) (2026)
- [Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operators](https://mlsanity.com/paper/2508.00643/) (2025)
- [Parametric Learning of Time-Advancement Operators for Unstable Flame Evolution](https://mlsanity.com/paper/2402.10238/) (2024)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [One model to solve them all: 2BSDE families via neural operators](https://mlsanity.com/paper/2511.01125/) (2025)
- [One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators](https://mlsanity.com/paper/2605.06873/) (2026)
- [Fourier Neural Operators for Learning Dynamics in Quantum Spin Systems](https://mlsanity.com/paper/2409.03302/) (2024)
- [Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEs](https://mlsanity.com/paper/2410.14788/) (2024)
- [PODNO: Proper Orthogonal Decomposition Neural Operators](https://mlsanity.com/paper/2504.18513/) (2025)
- [Learning High-dimensional Ionic Model Dynamics Using Fourier Neural Operators](https://mlsanity.com/paper/2505.14039/) (2025)
- [Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictors](https://mlsanity.com/paper/2509.26443/) (2025)
- [Delay-adaptive Control of Nonlinear Systems with Approximate Neural Operator Predictors](https://mlsanity.com/paper/2508.20367/) (2025)
- [Geometric Autoencoder Priors for Bayesian Inversion: Learn First Observe Later](https://mlsanity.com/paper/2509.19929/) (2025)
- [Efficient High-Accuracy PDEs Solver with the Linear Attention Neural Operator](https://mlsanity.com/paper/2510.16816/) (2025)
- [Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems](https://mlsanity.com/paper/2602.11208/) (2026)
- [Operator Learning for Smoothing and Forecasting](https://mlsanity.com/paper/2603.20359/) (2026)
- [Zero-shot generalization of transformer neural operators to larger domains](https://mlsanity.com/paper/2606.14597/) (2026)
- [Enforcing Dirichlet Boundary Conditions in Operator Learning](https://mlsanity.com/paper/2608.27256/) (2026)
- [Reformulating Neural Operators in d+1 Dimensions for Embedding Evolution](https://mlsanity.com/paper/2505.11766/) (2025)
- [Deep Micro Solvers for Rough-Wall Stokes Flow in a Heterogeneous Multiscale Method](https://mlsanity.com/paper/2507.13902/) (2025)
- [Bulk-boundary decomposition of neural networks](https://mlsanity.com/paper/2511.02003/) (2025)

## Abstract (arXiv)

Neural operator architectures approximate operators between infinite-dimensional Banach spaces of functions. They are gaining increased attention in computational science and engineering, due to their potential both to accelerate traditional numerical methods and to enable data-driven discovery. As the field is in its infancy basic questions about minimal requirements for universal approximation remain open. It is clear that any general approximation of operators between spaces of functions must be both nonlocal and nonlinear. In this paper we describe how these two attributes may be combined in a simple way to deduce universal approximation. In so doing we unify the analysis of a wide range of neural operator architectures and open up consideration of new ones. A popular variant of neural operators is the Fourier neural operator (FNO). Previous analysis proving universal operator approximation theorems for FNOs resorts to use of an unbounded number of Fourier modes, relying on intuition from traditional analysis of spectral methods. The present work challenges this point of view: (i) the work reduces FNO to its core essence, resulting in a minimal architecture termed the ``averaging neural operator'' (ANO); and (ii) analysis of the ANO shows that even this minimal ANO architecture benefits from universal approximation. This result is obtained based on only a spatial average as its only nonlocal ingredient (corresponding to retaining only a single Fourier mode in the special case of the FNO). The analysis paves the way for a more systematic exploration of nonlocality, both through the development of new operator learning architectures and the analysis of existing and new architectures. Numerical results are presented which give insight into complexity issues related to the roles of channel width (embedding dimension) and number of Fourier modes.