mlsanityData to 14 Oct 2026

Size Lowerbounds for Deep Operator Networks

arXiv 2308.06338 ↗First posted 11 Aug 2023cs.LG cs.CC cs.NAcited by 6

Not on the map. Why?

Abstract (arXiv)

Deep Operator Networks are an increasingly popular paradigm for solving regression in infinite dimensions and hence solve families of PDEs in one shot. In this work, we aim to establish a first-of-its-kind data-dependent lowerbound on the size of DeepONets required for them to be able to reduce empirical error on noisy data. In particular, we show that for low training errors to be obtained on n data points it is necessary that the common output dimension of the branch and the trunk net be scaling as Omega left ( √[leftroot-1uproot-14]n right ). This inspires our experiments with DeepONets solving the advection-diffusion-reaction PDE, where we demonstrate the possibility that at a fixed model size, to leverage increase in this common output dimension and get monotonic lowering of training error, the size of the training data might necessarily need to scale at least quadratically with it.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 31 of its 61 references that are in the map. First posted on arXiv 11 Aug 2023.

Builds on

31 of 61 references.

References · 31

  1. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  2. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  3. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 776
  4. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  5. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  6. 2019DeepXDE: A deep learning library for solving differential equationscited by 619
  7. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 608
  8. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 427
  9. 2020B-PINNs: Bayesian Physics-Informed Neural Networks for Forward and Inverse PDE Problems with Noisy Datacited by 335
  10. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  11. 2023Spherical Fourier Neural Operators: Learning Stable Dynamics on the Spherecited by 233
  12. 2017Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equationscited by 208
19 more
  1. 2022FourCastNet: Accelerating Global High-Resolution Weather Forecasting using Adaptive Fourier Neural Operatorscited by 188
  2. 2018To understand deep learning we need to understand kernel learningcited by 187
  3. 2021A Universal Law of Robustness via Isoperimetrycited by 151
  4. 2018Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolatecited by 99
  5. 2019A Model of Double Descent for High-dimensional Binary Linear Classificationcited by 81
  6. 2022Wavelet neural operator: a neural operator for parametric partial differential equationscited by 78
  7. 2022Generic bounds on the approximation error for physics-informed (and) operator learningcited by 65
  8. 2018Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Datacited by 36
  9. 2020Analytic Study of Double Descent in Binary Classification: The Impact of Losscited by 28
  10. 2022Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equationscited by 28
  11. 2023Learning in latent spaces improves the predictive accuracy of deep neural operatorscited by 19
  12. 2020Understanding Generalization in Deep Learning via Tensor Methodscited by 14
  13. 2022MultiAuto-DeepONet: A Multi-resolution Autoencoder DeepONet for Nonlinear Dimension Reduction, Uncertainty Quantification and Operator Learning of Forward and Inverse Stochastic Problemscited by 13
  14. 2022Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functionscited by 11
  15. 2023Fourier Neural Operator Surrogate Model to Predict 3D Seismic Waves Propagationcited by 11
  16. 2023Sparsity-aware generalization theory for deep neural networkscited by 8
  17. 2022Towards Size-Independent Generalization Bounds for Deep Operator Netscited by 6
  18. 2023On Size-Independent Sample Complexity of ReLU Networkscited by 6
  19. 2023Deep Learning and Computational Physics (Lecture Notes)cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 1.

6 papers.

  1. 2024Operator Learning: Algorithms and Analysiscited by 85
  2. 2024Adaptive Neural-Operator Backstepping Control of a Benchmark Hyperbolic PDEcited by 8
  3. 2026Diagnosing Failure Modes of Neural Operators Across Diverse PDE Familiescited by 6
  4. 2025Noisy PDE Training Requires Bigger PINNscited by 2
  5. 2025Stabilization of nonlinear systems with unknown delays via delay-adaptive neural operator approximate predictorscited by 2
  6. 2025Delay-adaptive Control of Nonlinear Systems with Approximate Neural Operator Predictorscited by 1