mlsanityData to 7 Oct 2026

Towards Size-Independent Generalization Bounds for Deep Operator Nets

arXiv 2205.11359 ↗First posted 23 May 2022cs.LG cs.NA math.NAcited by 6

Not on the map. Why?

Abstract (arXiv)

In recent times machine learning methods have made significant advances in becoming a useful tool for analyzing physical systems. A particularly active area in this theme has been "physics-informed machine learning" which focuses on using neural nets for numerically solving differential equations. In this work, we aim to advance the theory of measuring out-of-sample error while training DeepONets - which is among the most versatile ways to solve P.D.E systems in one-shot. Firstly, for a class of DeepONets, we prove a bound on their Rademacher complexity which does not explicitly scale with the width of the nets involved. Secondly, we use this to show how the Huber loss can be chosen so that for these DeepONet classes generalization error bounds can be obtained that have no explicit dependence on the size of the nets. The effective capacity measure for DeepONets that we thus derive is also shown to correlate with the behavior of generalization error in experiments.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 25 of its 59 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

25 of 59 references.

References · 25

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  5. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  6. 2021Learning the solution operator of parametric partial differential equations with physics-informed DeepOnetscited by 537
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  8. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  9. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  10. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  11. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  12. 2020Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEscited by 115
13 more
  1. 2020Rethinking Bias-Variance Trade-off for Generalization of Neural Networkscited by 107
  2. 2022Deep transfer operator learning for partial differential equations under conditional shiftcited by 83
  3. 2020Algorithms for Solving High Dimensional PDEs: From Nonlinear Monte Carlo to Machine Learningcited by 77
  4. 2019Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Predictioncited by 67
  5. 2017Data-driven discovery of Koopman eigenfunctions for controlcited by 64
  6. 2021A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learningcited by 54
  7. 2021Neural Stochastic PDEs: Resolution-Invariant Learning of Continuous Spatiotemporal Dynamicscited by 46
  8. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
  9. 2020Dropout: Explicit Forms and Capacity Controlcited by 27
  10. 2020Teaching the Incompressible Navier-Stokes Equations to Fast Neural Surrogate Models in 3Dcited by 26
  11. 2022MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problemscited by 13
  12. 2022Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functionscited by 11
  13. 2021A Study of the Mathematics of Deep Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 2; 2025: 1; 2026: 1.

6 papers.

  1. 2022Deep Learning Methods for Partial Differential Equations and Related Parameter Identification Problemscited by 34
  2. 2025Operator Learning: A Statistical Perspectivecited by 14
  3. 2023Size Lowerbounds for Deep Operator Networkscited by 6
  4. 2024Separable Operator Networkscited by 4
  5. 2024DeepONet for Solving Nonlinear Partial Differential Equations with Physics-Informed Trainingcited by 1
  6. 2026Generalization Error Bounds for Picard-Type Operator Learning in Nonlinear Parabolic PDEscited by 0