# Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transform

arXiv: https://arxiv.org/abs/2408.10996 · HTML: https://mlsanity.com/paper/2408.10996/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 65 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 15 of 15)

- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Characterization of the Variation Spaces Corresponding to Shallow Neural Networks](https://mlsanity.com/paper/2106.15002/) (2021)
- [Nonparametric regression using over-parameterized shallow ReLU neural networks](https://mlsanity.com/paper/2306.08321/) (2023)
- [Ridges, Neural Networks, and the Radon Transform](https://mlsanity.com/paper/2203.02543/) (2022)
- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)
- [Rates of Approximation by ReLU Shallow Neural Networks](https://mlsanity.com/paper/2307.12461/) (2023)
- [Kernel based analysis of massive data](https://mlsanity.com/paper/2003.13226/) (2020)
- [Why Shallow Networks Struggle to Approximate and Learn High Frequencies](https://mlsanity.com/paper/2306.17301/) (2023)
- [Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks](https://mlsanity.com/paper/2307.15285/) (2023)
- [Tractability of approximation by general shallow networks](https://mlsanity.com/paper/2308.03230/) (2023)

## Built on by (most cited first, 6 of 6)

- [Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces](https://mlsanity.com/paper/2501.11275/) (2025)
- [Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connections](https://mlsanity.com/paper/2504.15110/) (2025)
- [A Gap Between the Gaussian RKHS and Neural Networks: An Infinite-Center Asymptotic Analysis](https://mlsanity.com/paper/2502.16331/) (2025)
- [Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk Networks on the Sphere](https://mlsanity.com/paper/2510.04060/) (2025)
- [A Gap Between Decision Trees and Neural Networks](https://mlsanity.com/paper/2601.03919/) (2026)
- [Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples](https://mlsanity.com/paper/2608.06687/) (2026)

## Abstract (arXiv)

Let Omegasubset ℝd be a bounded domain. We consider the problem of how efficiently shallow neural networks with the ReLUk activation function can approximate functions from Sobolev spaces Ws(Lp(Omega)) with error measured in the Lq(Omega)-norm. Utilizing the Radon transform and recent results from discrepancy theory, we provide a simple proof of nearly optimal approximation rates in a variety of cases, including when q≤ p, p≥ 2, and s ≤ k + (d+1)/2. The rates we derive are optimal up to logarithmic factors, and significantly generalize existing results. An interesting consequence is that the adaptivity of shallow ReLUk neural networks enables them to obtain optimal approximation rates for smoothness up to order s = k + (d+1)/2, even though they represent piecewise polynomials of fixed degree k.