Data to 14 Oct 2026

Deep Nonparametric Estimation of Operators between Infinite Dimensional Spaces

arXiv 2201.00217 ↗First posted 1 Jan 2022stat.ML cs.LGcited by 45

Not on the map. Why?

Abstract (arXiv)

Learning operators between infinitely dimensional spaces is an important learning task arising in wide applications in machine learning, imaging science, mathematical modeling and simulations, etc. This paper studies the nonparametric estimation of Lipschitz operators using deep neural networks. Non-asymptotic upper bounds are derived for the generalization error of the empirical risk minimizer over a properly chosen network class. Under the assumption that the target operator exhibits a low dimensional structure, our error bounds decay as the training sample size increases, with an attractive fast rate depending on the intrinsic dimension in our estimation. Our assumptions cover most scenarios in real applications and our results give rise to fast rates by exploiting low dimensional structures of data in operator estimation. We also investigate the influence of network structures (e.g., network width, depth, and sparsity) on the generalization error of the neural network estimator and propose a general suggestion on the choice of network structures to maximize the learning efficiency quantitatively.

Key facts. Cited by 45 papers in the map (data through 14 Oct 2026). Builds on 41 of its 99 references that are in the map. First posted on arXiv 1 Jan 2022.

Builds on

41 of 99 references.

References · 41

2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 2,018
2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 734
2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
2016Error bounds for approximations with deep ReLU networkscited by 574
2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
2018Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantificationcited by 199
2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
2020Deep Network Approximation for Smooth Functionscited by 164

29 more

2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 162
2019Deep Learning on Image Denoising: An overviewcited by 128
2019Deep Network Approximation Characterized by Number of Neuronscited by 122
2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 116
2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 96
2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
2019The phase diagram of approximation rates for deep neural networkscited by 88
2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
2019Deep ReLU network approximation of functions on a manifoldcited by 74
2019Nonparametric Regression on Low-Dimensional Manifolds using Deep ReLU Networks : Function Approximation and Statistical Recoverycited by 73
2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
2018Deep Neural Networks for Estimation and Inferencecited by 62
2015Provable approximation properties for deep neural networkscited by 60
2021Convergence Rates for Learning Linear Operators from Noisy Datacited by 55
2018Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equationscited by 45
2020Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theorycited by 41
2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
2021Elementary superexpressive activationscited by 36
2018SwitchNet: a neural network model for forward and inverse scattering problemscited by 35
2021Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networkscited by 32
2021A Priori Generalization Error Analysis of Two-Layer Neural Networks for Solving High Dimensional Schr"odinger Eigenvalue Problemscited by 19
2018BCR-Net: a neural network based on the nonstandard wavelet formcited by 18
2021A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equationscited by 18
2019Learning to Synthesize: Robust Phase Retrieval at Low Photon countscited by 16
2021The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimationcited by 14
2020A deep network construction that adapts to intrinsic dimensionality beyond the domaincited by 10
2021Stationary Density Estimation of Ito Diffusions Using Deep Learningcited by 9
2020Doubly Robust Off-Policy Learning on Low-Dimensional Manifolds by Deep Neural Networkscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 7; 2024: 13; 2025: 12; 2026: 6.

45 papers.

Sort:

2024Operator Learning: Algorithms and Analysiscited by 85
2023Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivativescited by 30
2024Data Complexity Estimates for Operator Learningcited by 26
2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
2022Neural and spectral operator surrogates: unified construction and expression rate boundscited by 21
2024Operator Learning Using Random Features: A Tool for Scientific Computingcited by 21
2023Deep Operator Learning Lessens the Curse of Dimensionality for PDEscited by 18
2025Operator Learning: A Statistical Perspectivecited by 17
2022IAE-Net: Integral Autoencoders for Discretization-Invariant Learningcited by 15
2024MODNO: Multi Operator Learning With Distributed Neural Operatorscited by 14
2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 14
2022Minimax Optimal Kernel Operator Learning via Multilevel Trainingcited by 13

33 more

2024Generalization Error Guaranteed Auto-Encoder-Based Nonlinear Model Reduction for Operator Learningcited by 12
2024Operator Learning of Lipschitz Operators: An Information-Theoretic Perspectivecited by 11
2024The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measurescited by 11
2023Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustnesscited by 10
2025A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theorycited by 9
2022Local approximation of operatorscited by 8
2022Approximation of Functionals by Neural Network without Curse of Dimensionalitycited by 8
2023Double-well Net for Image Segmentationcited by 7
2024Nonlinear functional regression by functional deep neural network with kernel embeddingcited by 7
2022Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraintcited by 6
2024Separable Operator Networkscited by 5
2024Controlling Statistical, Discretization, and Truncation Errors in Learning Fourier Linear Operatorscited by 5
2025Cauchy Random Features for Operator Learning in Sobolev Spacecited by 5
2025Towards Sharp Minimax Risk Bounds for Operator Learningcited by 5
2023Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Normscited by 4
2025Theory-to-Practice Gap for Neural Networks and Neural Operatorscited by 4
2025New universal operator approximation theorem for encoder-decoder architecturescited by 4
2026A short tour of operator learning theory: Convergence rates, statistical limits, and open questionscited by 4
2026Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learningcited by 4
2023Connections between Operator-splitting Methods and Deep Neural Networks with Applications in Image Segmentationcited by 3
2025Coefficient-to-Basis Network: A Fine-Tunable Operator Learning Framework for Inverse Problems with Adaptive Discretizations and Theoretical Guaranteescited by 3
2025Deep Neural Operator Learning for Probabilistic Modelscited by 3
2026Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networkscited by 3
2023On the Approximation of Bi-Lipschitz Maps by Invertible Neural Networkscited by 2
2024Polynomial Scaling is Possible For Neural Operator Approximations of Structured Families of BSDEscited by 2
2025Learn Singularly Perturbed Solutions via Homotopy Dynamicscited by 2
2024DeepONet for Solving Nonlinear Partial Differential Equations with Physics-Informed Trainingcited by 1
2025Curse of Dimensionality in Neural Network Optimizationcited by 1
2025A Kernel-based Stochastic Approximation Framework for Nonlinear Operator Learningcited by 1
2026Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalizationcited by 1
2025Multi-Scale Finite Expression Method for PDEs with Oscillatory Solutions on Complex Domainscited by 0
2026Cost-Accuracy Trade-offs: Neural Operator vs Classical Numerical Solvercited by 0
2026Neural Scaling Laws of Transformer Operator Networkcited by 0