mlsanityData to 7 Oct 2026

Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transform

arXiv 2408.10996 ↗First posted 20 Aug 2024stat.ML cs.LG cs.NAcited by 6

Not on the map. Why?

Abstract (arXiv)

Let Omegasubset ℝd be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLUk activation function can approximate functions from Sobolev spaces Ws(Lp(Omega)) with error measured in the Lq(Omega)-norm. Utilizing the Radon transform and recent results from discrepancy theory, we provide a simple proof of nearly optimal approximation rates in a variety of cases, including when q≤ p, p≥ 2, and s ≤ k + (d+1)/2. The rates we derive are optimal up to logarithmic factors, and significantly generalize existing results. An interesting consequence is that the adaptivity of shallow ReLUk neural networks enables them to obtain optimal approximation rates for smoothness up to order s = k + (d+1)/2, even though they represent piecewise polynomials of fixed degree k.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 15 of its 65 references that are in the map. First posted on arXiv 20 Aug 2024.

Builds on

15 of 65 references.

References · 15

  1. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  2. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  3. 2019A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Casecited by 113
  4. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  5. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  6. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  7. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  8. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  9. 2022Ridges, Neural Networks, and the Radon Transformcited by 18
  10. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  11. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  12. 2020Kernel based analysis of massive datacited by 11
3 more
  1. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  2. 2023Why Shallow Networks Struggle to Approximate and Learn High Frequenciescited by 10
  3. 2023Tractability of approximation by general shallow networkscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  2. 2025A Gap Between the Gaussian RKHS and Neural Networks: An Infinite-Center Asymptotic Analysiscited by 2
  3. 2025Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connectionscited by 2
  4. 2025Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk Networks on the Spherecited by 1
  5. 2026A Gap Between Decision Trees and Neural Networkscited by 0
  6. 2026Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samplescited by 0