# Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

arXiv: https://arxiv.org/abs/2608.06687 · HTML: https://mlsanity.com/paper/2608.06687/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 96 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 19 of 19)

- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [A Priori Estimates of the Population Risk for Two-layer Neural Networks](https://mlsanity.com/paper/1810.06397/) (2018)
- [Representation formulas and pointwise properties for Barron functions](https://mlsanity.com/paper/2006.05982/) (2020)
- [Characterization of the Variation Spaces Corresponding to Shallow Neural Networks](https://mlsanity.com/paper/2106.15002/) (2021)
- [On the distance between two neural networks and the stability of learning](https://mlsanity.com/paper/2002.03432/) (2020)
- [Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks](https://mlsanity.com/paper/2109.08844/) (2021)
- [Theory of Deep Convolutional Neural Networks II: Spherical Analysis](https://mlsanity.com/paper/2007.14285/) (2020)
- [An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions](https://mlsanity.com/paper/2309.07049/) (2023)
- [DeepSphere: towards an equivariant graph-based spherical CNN](https://mlsanity.com/paper/1904.05146/) (2019)
- [Kernel based analysis of massive data](https://mlsanity.com/paper/2003.13226/) (2020)
- [Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions](https://mlsanity.com/paper/1709.08174/) (2017)
- [Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transform](https://mlsanity.com/paper/2408.10996/) (2024)
- [Tractability of approximation by general shallow networks](https://mlsanity.com/paper/2308.03230/) (2023)
- [Spherical Analysis of Learning Nonlinear Functionals](https://mlsanity.com/paper/2410.01047/) (2024)

## Abstract (arXiv)

We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations mathfrak Lβ u=f using linearized ReLUk neural networks on the sphere, where mathfrak Lβ is a positive elliptic spectral multiplier of order β. Given a parameter set Thetan=θj^*j=1nsubsetmathbb Sd, we approximate u in the linearized network space Lnk(Thetan) by the discrete residual on the collocation points ηi^*i=1m beginequation* un,m∈argminvn∈ Lnk(Thetan)frac1msumi=1mleft(f(ηi^*)-mathfrak Lβ vn(ηi^*)right)2. endequation* With k>fracd-12+β, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with mgtrsim n, we prove that beginequation* |u-un,m|_mathcal Hβ(mathbb Sd)eqsim|f-mathfrak Lβ un,m|mathcal L2(mathbb Sd)lesssim n^-fracrd begincases |f|_mathcal Wr,p(mathbb Sd),&fracdp<r≤ fracd2,~p>2, |f|mathcal Hr(mathbb Sd),&r>fracd2. endcases endequation* We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d. uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLUk network spaces. If underline h denotes the antipodal separation distance of the network parameters, then beginequation* |vn|mathcal Hr(mathbb Sd)lesssimunderline h-(r-s)|vn|mathcal Hs(mathbb Sd),qquad 0≤ s<r<k+tfrac12. endequation*