Sampling Complexity of Deep Approximation Spaces
Not on the map. Why?
Abstract (arXiv)
While it is well-known that neural networks enjoy excellent approximation capabilities, it remains a big challenge to compute such approximations from point samples. Based on tools from Information-based complexity, recent work by Grohs and Voigtlaender [Journal of the FoCM (2023)] developed a rigorous framework for assessing this so-called "theory-to-practice gap". More precisely, in that work it is shown that there exist functions that can be approximated by neural networks with ReLU activation function at an arbitrary rate while requiring an exponentially growing (in the input dimension) number of samples for their numerical computation. The present study extends these findings by showing analogous results for the ReQU activation function.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 20 Dec 2023.
Builds on
21 of 30 references.
References · 21
- 2016Error bounds for approximations with deep ReLU networkscited by 574
- 2017Solving high-dimensional partial differential equations using deep learningcited by 556
- 2017Size-Independent Sample Complexity of Neural Networkscited by 290
- 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
- 2017A unified deep artificial neural network approach to partial differential equations in complex geometriescited by 122
- 2019Deep Neural Network Approximation Theorycited by 93
- 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 91
- 2019A Theoretical Analysis of Deep Neural Networks and Parametric PDEscited by 74
- 2019Approximation spaces of deep neural networkscited by 51
- 2018Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black-Scholes Partial Differential Equationscited by 45
- 2018Solving the Kolmogorov PDE by means of deep learningcited by 41
- 2021Proof of the Theory-to-Practice Gap in Deep Learning via Sampling Complexity bounds for Neural Network Approximation Spacescited by 30
9 more
- 2020Deep neural network approximation for high-dimensional elliptic PDEs with boundary conditionscited by 25
- 2018Neural Networks Trained to Solve Differential Equations Learn General Representationscited by 16
- 2019Deep Neural Network Framework Based on Backward Stochastic Differential Equations for Pricing and Hedging American Options in High Dimensionscited by 14
- 2020Approximations with deep neural networks in Sobolev time-spacecited by 13
- 2019Uniform error estimates for artificial neural network approximations for heat equationscited by 11
- 2019Deep Neural Networks for Rotation-Invariance Approximation and Learningcited by 10
- 2018Construction of neural networks for realization of localized deep learningcited by 7
- 2019Space-time error estimates for deep neural network approximations for differential equationscited by 6
- 2023Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spacescited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.