mlsanityData to 7 Oct 2026

Rates of Approximation by ReLU Shallow Neural Networks

arXiv 2307.12461 ↗First posted 24 Jul 2023cs.LG stat.MLcited by 12

Not on the map. Why?

Abstract (arXiv)

Neural networks activated by the rectified linear unit (ReLU) play a central role in the recent development of deep learning. The topic of approximating functions from H"older spaces by these networks is crucial for understanding the efficiency of the induced learning algorithms. Although the topic has been well investigated in the setting of deep neural networks with many layers of hidden neurons, it is still open for shallow networks having only one hidden layer. In this paper, we provide rates of uniform approximation by these networks. We show that ReLU shallow neural networks with m hidden neurons can uniformly approximate functions from the H"older space W_∞r([-1, 1]d) with rates O((log m)^frac12 +dm^-fracrdfracd+2d+4) when r<d/2 +2. Such rates are very close to the optimal one O(m^-fracrd) in the sense that fracd+2d+4 is close to 1, when the dimension d is large.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 14 of its 41 references that are in the map. First posted on arXiv 24 Jul 2023.

Builds on

14 of 41 references.

References · 14

  1. 2016Error bounds for approximations with deep ReLU networkscited by 567
  2. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  3. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
  4. 2018Universality of Deep Convolutional Neural Networkscited by 176
  5. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  6. 2017Optimal Approximation with Sparsely Connected Deep Neural Networkscited by 90
  7. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  8. 2016Deep vs. shallow networks : An approximation theory perspectivecited by 63
  9. 2015Provable approximation properties for deep neural networkscited by 60
  10. 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
  11. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  12. 2020Depth Selection for Deep ReLU Nets in Feature Extraction and Generalizationcited by 13
2 more
  1. 2017Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functionscited by 8
  2. 2019Dimension independent bounds for general shallow networkscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 5; 2026: 2.

12 papers.

  1. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 8
  2. 2024Universal approximation results for neural networks with non-polynomial activation function over non-compact domainscited by 7
  3. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6
  4. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  5. 2025On the Expressive Power of Mixture-of-Experts for Structured Complex Taskscited by 4
  6. 2024Approximation Rates and VC-Dimension Bounds for (P)ReLU MLP Mixture of Expertscited by 2
  7. 2024Approximation Bounds for Recurrent Neural Networks with Application to Regressioncited by 2
  8. 2025Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk Networks on the Spherecited by 1
  9. 2025Error analysis for the deep Kolmogorov methodcited by 0
  10. 2025Generalization Analysis for Classification on Korobov Spacecited by 0
  11. 2026Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Widthcited by 0
  12. 2026Shallow neural network approximation in mixed Sobolev spacescited by 0