mlsanityData to 14 Oct 2026

Arbitrary-Depth Universal Approximation Theorems for Operator Neural Networks

arXiv 2109.11354 ↗First posted 23 Sep 2021cs.LG cs.NA math.CAcited by 12

Not on the map. Why?

Abstract (arXiv)

The standard Universal Approximation Theorem for operator neural networks (NNs) holds for arbitrary width and bounded depth. Here, we prove that operator NNs of bounded width and arbitrary depth are universal approximators for continuous nonlinear operators. In our main result, we prove that for non-polynomial activation functions that are continuously differentiable at a point with a nonzero derivative, one can construct an operator NN of width five, whose inputs are real numbers with finite decimal representations, that is arbitrarily close to any given continuous nonlinear operator. We derive an analogous result for non-affine polynomial activation functions. We also show that depth has theoretical advantages by constructing operator ReLU NNs of depth 2k3+8 and constant width that cannot be well-approximated by any operator ReLU NN of depth k, unless its width is exponential in k.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 10 of its 16 references that are in the map. First posted on arXiv 23 Sep 2021.

Builds on

10 of 16 references.

References · 10

  1. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
  2. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  3. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 320
  4. 2016Benefits of depth in neural networkscited by 250
  5. 2019Universal Approximation with Deep Narrow Networkscited by 200
  6. 2015The Power of Depth for Feedforward Neural Networkscited by 183
  7. 2017Solving ill-posed inverse problems using iterative deep neural networkscited by 145
  8. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 122
  9. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  10. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 1; 2024: 0; 2025: 7.

12 papers.

  1. 2022Physics-Informed Deep Neural Operator Networkscited by 113
  2. 2022Physics-Informed Machine Learning: A Survey on Problems, Methods and Applicationscited by 92
  3. 2021Improved architectures and training algorithms for deep operator networkscited by 86
  4. 2022Neural and spectral operator surrogates: unified construction and expression rate boundscited by 21
  5. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  6. 2025Geometry and Optimization of Shallow Polynomial Networkscited by 10
  7. 2025Block-Biased Mamba for Long-Range Sequence Processingcited by 8
  8. 2025One model to solve them all: 2BSDE families via neural operatorscited by 4
  9. 2023An Operator Learning Framework for Spatiotemporal Super-resolution of Scientific Simulationscited by 0
  10. 2025Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flowscited by 0
  11. 2025Algorithms and data structures for automatic precision estimation of neural networkscited by 0
  12. 2025Hybrid twinning using PBDW and DeepONet for the effective state estimation and prediction on partially known systemscited by 0