mlsanityData to 14 Oct 2026

Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

arXiv 2609.19937 ↗First posted 17 Sep 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that smooth functions can be well approximated by ReLU neural networks with path norm constraint on the weights. We extend these results from uniform approximation to approximation in Sobolev norm. Specifically, we analyze how well Sobolev functions in Wn,p can be approximated by neural networks with width W, depth L and path norm bounded by K, when the approximation error is measured in the W1,p-norm. For shallow networks with depth L=1, we derive the approximation error bound O(maxW-(n-1)/d, K-(n-1)/(s-n)), when the smoothness index satisfies n<s=(d+3)/2 and the input is d-dimensional. For deep networks, we remove the restriction on the smoothness by showing that the approximation bound O(K-(n-1)/(d+d/p+1)) holds if the width W and depth L are sufficiently large.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 70 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

39 of 70 references.

References · 39

  1. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  2. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 678
  3. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 583
  4. 2016Error bounds for approximations with deep ReLU networkscited by 574
  5. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  6. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  7. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  8. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  9. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 263
  10. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  11. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  12. 2020Deep Network Approximation for Smooth Functionscited by 164
27 more
  1. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  2. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 96
  3. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  4. 2019Deep Neural Network Approximation Theorycited by 93
  5. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 91
  6. 2020The Kolmogorov-Arnold representation theorem revisitedcited by 89
  7. 2019The phase diagram of approximation rates for deep neural networkscited by 88
  8. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  9. 2021On the approximation of functions by tanh neural networkscited by 81
  10. 2018A Priori Estimates of the Population Risk for Two-layer Neural Networkscited by 76
  11. 2019A Theoretical Analysis of Deep Neural Networks and Parametric PDEscited by 74
  12. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  13. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 51
  14. 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 49
  15. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  16. 2023Nearly Optimal VC-Dimension and Pseudo-Dimension Bounds for Deep Neural Network Derivativescited by 30
  17. 2022Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activationscited by 29
  18. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  19. 2022Approximation bounds for norm constrained neural networks with applications to regression and GANscited by 22
  20. 2021A Priori Generalization Error Analysis of Two-Layer Neural Networks for Solving High Dimensional Schr"odinger Eigenvalue Problemscited by 19
  21. 2024On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networkscited by 13
  22. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  23. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  24. 2023Tractability of approximation by general shallow networkscited by 4
  25. 2024DRM Revisited: A Complete Error Analysiscited by 4
  26. 2026Learning Sparse Compositional Functions with Norm-Constrained Neural Networkscited by 3
  27. 2026Shallow ReLUs Networks in Lp-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalizationcited by 2

Built on by

Not cited yet.