Condition Numbers and Eigenvalue Spectra of Shallow Networks on Spheres
Not on the map. Why?
Abstract (arXiv)
We present an estimation of the condition numbers of the mass and stiffness matrices arising from shallow ReLUk neural networks defined on the unit sphere~mathbbSd. In particular, when θj^*j=1n subset mathbbSd is antipodally quasi-uniform, the condition number is sharp. Indeed, in this case, we obtain sharp asymptotic estimates for the full spectrum of eigenvalues and characterize the structure of the corresponding eigenspaces, showing that the smallest eigenvalues are associated with an eigenbasis of low-degree polynomials while the largest eigenvalues are linked to high-degree polynomials. This spectral analysis establishes a precise correspondence between the approximation power of the network and its numerical stability.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 72 references that are in the map. First posted on arXiv 4 Nov 2025.
Builds on
24 of 72 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
- 2018On the Spectral Bias of Neural Networkscited by 1,341
- 2019The generalization error of random features regression: Precise asymptotics and double descent curvecited by 413
- 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 368
- 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
- 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
- 2019Two models of double descent for weak featurescited by 195
- 2016Generalization Properties of Learning with Random Featurescited by 164
- 2015On the Equivalence between Kernel Quadrature Rules and Random Feature Expansionscited by 138
- 2020Generalisation error in learning with random features and the hidden manifold modelcited by 127
12 more
- 2020The Random Feature Model for Input-Output Maps between Banach Spacescited by 90
- 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
- 2020Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equationscited by 81
- 2019Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equationscited by 67
- 2022On the Activation Function Dependence of the Spectral Bias of Neural Networkscited by 28
- 2021Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networkscited by 21
- 2014Randomized Nonlinear Component Analysiscited by 18
- 2016Deep Learning in Financecited by 18
- 2023TransNet: Transferable Neural Networks for Partial Differential Equationscited by 13
- 2023Why Shallow Networks Struggle to Approximate and Learn High Frequenciescited by 11
- 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
- 2025A Multiple Transferable Neural Network Method with Domain Decomposition for Elliptic Interface Problemscited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.