# Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces

arXiv: https://arxiv.org/abs/2201.00217 · HTML: https://mlsanity.com/paper/2201.00217/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 99 references that are in the map.
- First posted on arXiv 1 Jan 2022.

## Builds on (most cited first, 30 of 41)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks](https://mlsanity.com/paper/1905.13210/) (2019)
- [Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification](https://mlsanity.com/paper/1801.06879/) (2018)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality](https://mlsanity.com/paper/1810.08033/) (2018)
- [Deep Learning on Image Denoising: An overview](https://mlsanity.com/paper/1912.13171/) (2019)
- [Deep Network Approximation Characterized by Number of Neurons](https://mlsanity.com/paper/1906.05497/) (2019)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs](https://mlsanity.com/paper/2006.16144/) (2020)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [The Random Feature Model for Input-Output Maps between Banach Spaces](https://mlsanity.com/paper/2005.10224/) (2020)
- [The phase diagram of approximation rates for deep neural networks](https://mlsanity.com/paper/1906.09477/) (2019)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [A Priori Estimates of the Population Risk for Two-layer Neural Networks](https://mlsanity.com/paper/1810.06397/) (2018)
- [Deep ReLU network approximation of functions on a manifold](https://mlsanity.com/paper/1908.00695/) (2019)
- [Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recovery](https://mlsanity.com/paper/1908.01842/) (2019)
- [Neural Network Approximation: Three Hidden Layers Are Enough](https://mlsanity.com/paper/2010.14075/) (2020)
- [Deep Neural Networks for Estimation and Inference](https://mlsanity.com/paper/1809.09953/) (2018)
- [Provable approximation properties for deep neural networks](https://mlsanity.com/paper/1509.07385/) (2015)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equations](https://mlsanity.com/paper/1809.03062/) (2018)
- [Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory](https://mlsanity.com/paper/2006.15733/) (2020)
- [Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neurons](https://mlsanity.com/paper/2107.02397/) (2021)

## Built on by (most cited first, 30 of 45)

- [Operator Learning: Algorithms and Analysis](https://mlsanity.com/paper/2402.15715/) (2024)
- [Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivatives](https://mlsanity.com/paper/2305.08466/) (2023)
- [Data Complexity Estimates for Operator Learning](https://mlsanity.com/paper/2405.15992/) (2024)
- [Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformer](https://mlsanity.com/paper/2201.13094/) (2022)
- [Neural and spectral operator surrogates: unified construction and expression rate bounds](https://mlsanity.com/paper/2207.04950/) (2022)
- [Operator Learning Using Random Features: A Tool for Scientific Computing](https://mlsanity.com/paper/2408.06526/) (2024)
- [Deep Operator Learning Lessens the Curse of Dimensionality for PDEs](https://mlsanity.com/paper/2301.12227/) (2023)
- [Operator Learning: A Statistical Perspective](https://mlsanity.com/paper/2504.03503/) (2025)
- [IAE-Net: Integral Autoencoders for Discretization-Invariant Learning](https://mlsanity.com/paper/2203.05142/) (2022)
- [MODNO: Multi Operator Learning With Distributed Neural Operators](https://mlsanity.com/paper/2404.02892/) (2024)
- [Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Study](https://mlsanity.com/paper/2410.00357/) (2024)
- [Minimax Optimal Kernel Operator Learning via Multilevel Training](https://mlsanity.com/paper/2209.14430/) (2022)
- [Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learning](https://mlsanity.com/paper/2401.10490/) (2024)
- [Operator Learning of Lipschitz Operators: An Information-Theoretic Perspective](https://mlsanity.com/paper/2406.18794/) (2024)
- [The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measures](https://mlsanity.com/paper/2410.23440/) (2024)
- [Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness](https://mlsanity.com/paper/2303.09863/) (2023)
- [A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theory](https://mlsanity.com/paper/2510.25379/) (2025)
- [Local approximation of operators](https://mlsanity.com/paper/2202.06392/) (2022)
- [Approximation of Functionals by Neural Network without Curse of Dimensionality](https://mlsanity.com/paper/2205.14421/) (2022)
- [Double-well Net for Image Segmentation](https://mlsanity.com/paper/2401.00456/) (2023)
- [Nonlinear functional regression by functional deep neural network with kernel embedding](https://mlsanity.com/paper/2401.02890/) (2024)
- [Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint](https://mlsanity.com/paper/2206.04569/) (2022)
- [Separable Operator Networks](https://mlsanity.com/paper/2407.11253/) (2024)
- [Controlling Statistical, Discretization, and Truncation Errors in Learning Fourier Linear Operators](https://mlsanity.com/paper/2408.09004/) (2024)
- [Cauchy Random Features for Operator Learning in Sobolev Space](https://mlsanity.com/paper/2503.00300/) (2025)
- [Towards Sharp Minimax Risk Bounds for Operator Learning](https://mlsanity.com/paper/2512.17805/) (2025)
- [Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Norms](https://mlsanity.com/paper/2311.04779/) (2023)
- [Theory-to-Practice Gap for Neural Networks and Neural Operators](https://mlsanity.com/paper/2503.18219/) (2025)
- [New universal operator approximation theorem for encoder-decoder architectures](https://mlsanity.com/paper/2503.24092/) (2025)
- [A short tour of operator learning theory: Convergence rates, statistical limits, and open questions](https://mlsanity.com/paper/2603.00819/) (2026)

## Abstract (arXiv)

Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc. This paper studies the nonparametric estimation of Lipschitz operators using deep neural networks. Non-asymptotic upper bounds are derived for the generalization error of the empirical risk minimizer over a properly chosen network class. Under the assumption that the target operator exhibits a low dimensional structure, our error bounds decay as the training sample size increases, with an attractive fast rate depending on the intrinsic dimension in our estimation. Our assumptions cover most scenarios in real applications and our results give rise to fast rates by exploiting low dimensional structures of data in operator estimation. We also investigate the influence of network structures (e.g., network width, depth, and sparsity) on the generalization error of the neural network estimator and propose a general suggestion on the choice of network structures to maximize the learning efficiency quantitatively.