mlsanityData to 14 Oct 2026

Nonlocality and Nonlinearity Implies Universality in Operator Learning

arXiv 2304.13221 ↗First posted 26 Apr 2023math.NA cs.LG cs.NAcited by 43

Not on the map. Why?

Abstract (arXiv)

Neural operator architectures approximate operators between infinite-dimensional Banach spaces of functions. They are gaining increased attention in computational science and engineering, due to their potential both to accelerate traditional numerical methods and to enable data-driven discovery. As the field is in its infancy basic questions about minimal requirements for universal approximation remain open. It is clear that any general approximation of operators between spaces of functions must be both nonlocal and nonlinear. In this paper we describe how these two attributes may be combined in a simple way to deduce universal approximation. In so doing we unify the analysis of a wide range of neural operator architectures and open up consideration of new ones. A popular variant of neural operators is the Fourier neural operator (FNO). Previous analysis proving universal operator approximation theorems for FNOs resorts to use of an unbounded number of Fourier modes, relying on intuition from traditional analysis of spectral methods. The present work challenges this point of view: (i) the work reduces FNO to its core essence, resulting in a minimal architecture termed the ``averaging neural operator'' (ANO); and (ii) analysis of the ANO shows that even this minimal ANO architecture benefits from universal approximation. This result is obtained based on only a spatial average as its only nonlocal ingredient (corresponding to retaining only a single Fourier mode in the special case of the FNO). The analysis paves the way for a more systematic exploration of nonlocality, both through the development of new operator learning architectures and the analysis of existing and new architectures. Numerical results are presented which give insight into complexity issues related to the roles of channel width (embedding dimension) and number of Fourier modes.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Builds on 24 of its 57 references that are in the map. First posted on arXiv 26 Apr 2023.

Builds on

24 of 57 references.

References · 24

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  4. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  5. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
  6. 2021Physics-Informed Neural Operator for Learning Partial Differential Equationscited by 527
  7. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  8. 2020Multipole Graph Neural Operator for Parametric Partial Differential Equationscited by 349
  9. 2021U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flowcited by 297
  10. 2020Model Reduction and Neural Networks for Parametric PDEscited by 258
  11. 2021Multiwavelet-based Operator Learning for Differential Equationscited by 234
  12. 2022MIONet: Learning multiple-input operators via tensor productcited by 181
12 more
  1. 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
  2. 2022NOMAD: Nonlinear Manifold Decoders for Operator Learningcited by 83
  3. 2022Learning Deep Implicit Fourier Neural Operators (IFNOs) with Applications to Heterogeneous Material Modelingcited by 65
  4. 2020A physics-informed operator regression framework for extracting data-driven continuum modelscited by 60
  5. 2021Convergence Rates for Learning Linear Operators from Noisy Datacited by 55
  6. 2023LNO: Laplace Neural Operator for Solving Differential Equationscited by 47
  7. 2021Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equationscited by 44
  8. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 42
  9. 2022Nonlinear Reconstruction for Operator Learning of PDEs with Discontinuitiescited by 30
  10. 2022Neural and spectral operator surrogates: unified construction and expression rate boundscited by 21
  11. 2022Variationally Mimetic Operator Networkscited by 19
  12. 2018Nonlinear integro-differential operator regression with neural networkscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 13; 2026: 19.

43 papers.

Sort:

  1. 2024Operator Learning: Algorithms and Analysiscited by 85
  2. 2024Continuum Attention for Neural Operatorscited by 40
  3. 2023Plasma Surrogate Modelling using Fourier Neural Operatorscited by 29
  4. 2023Deep Operator Network Approximation Rates for Lipschitz Operatorscited by 24
  5. 2024Active Learning for Neural PDE Solverscited by 22
  6. 2024Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problemscited by 13
  7. 2024The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measurescited by 11
  8. 2024LazyDINO: Fast, scalable, and efficiently amortized Bayesian inversion via structure-exploiting and surrogate-driven measure transportcited by 7
  9. 2025KANO: Kolmogorov-Arnold Neural Operatorcited by 7
  10. 2026Performance of Neural and Polynomial Operator Surrogatescited by 7
  11. 2025Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operatorscited by 6
  12. 2024Parametric Learning of Time-Advancement Operators for Unstable Flame Evolutioncited by 5
31 more
  1. 2025New universal operator approximation theorem for encoder-decoder architecturescited by 4
  2. 2025One model to solve them all: 2BSDE families via neural operatorscited by 4
  3. 2026One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operatorscited by 4
  4. 2024Fourier Neural Operators for Learning Dynamics in Quantum Spin Systemscited by 2
  5. 2024Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEscited by 2
  6. 2025PODNO: Proper Orthogonal Decomposition Neural Operatorscited by 2
  7. 2025Learning High-dimensional Ionic Model Dynamics Using Fourier Neural Operatorscited by 2
  8. 2025Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictorscited by 2
  9. 2025Delay-adaptive Control of Nonlinear Systems with Approximate Neural Operator Predictorscited by 1
  10. 2025Geometric Autoencoder Priors for Bayesian Inversion: Learn First Observe Latercited by 1
  11. 2025Efficient High-Accuracy PDEs Solver with the Linear Attention Neural Operatorcited by 1
  12. 2026Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systemscited by 1
  13. 2026Operator Learning for Smoothing and Forecastingcited by 1
  14. 2026Zero-shot generalization of transformer neural operators to larger domainscited by 1
  15. 2026Enforcing Dirichlet Boundary Conditions in Operator Learningcited by 1
  16. 2025Reformulating Neural Operators in d+1 Dimensions for Embedding Evolutioncited by 0
  17. 2025Deep Micro Solvers for Rough-Wall Stokes Flow in a Heterogeneous Multiscale Methodcited by 0
  18. 2025Bulk-boundary decomposition of neural networkscited by 0
  19. 2026Translation Invariance of Neural Operators for the FitzHugh-Nagumo Modelcited by 0
  20. 2026Predictor-Based Output-Feedback Control of Linear Systems with Time-Varying Input and Measurement Delays via Neural-Approximated Prediction Horizonscited by 0
  21. 2026Sampling-Horizon Neural Operator Predictors for Nonlinear Control under Delayed Inputscited by 0
  22. 2026Lotka-Sharpe Neural Operators for Control of Population PDEscited by 0
  23. 2026Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEscited by 0
  24. 2026Approximation of Maximally Monotone Operators : A Graph Convergence Perspectivecited by 0
  25. 2026Universal Approximation of Nonlinear Operators and Their Derivativescited by 0
  26. 2026Fourier Neural Operators with rank-1 lattice points and hyperbolic crosscited by 0
  27. 2026NeuralChaos: Optimal Adapted Approximation of Square Integrable Predictable Processescited by 0
  28. 2026Inertial Manifold Neural Operator for Dissipative Time-Dependent Partial Differential Equationscited by 0
  29. 2026Cost-Accuracy Trade-offs: Neural Operator vs Classical Numerical Solvercited by 0
  30. 2026Does Transolver really need a Transformer?cited by 0
  31. 2026Cluster Attention Neural Operators for Solving Parametric Partial Differential Equationscited by 0