mlsanityData to 7 Oct 2026

Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernels

arXiv 2005.10807 ↗First posted 21 May 2020math.FA cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

We establish a scale separation of Kolmogorov width type between subspaces of a given Banach space under the condition that a sequence of linear maps converges much faster on one of the subspaces. The general technique is then applied to show that reproducing kernel Hilbert spaces are poor L2-approximators for the class of two-layer neural networks in high dimension, and that multi-layer networks with small path norm are poor approximators for certain Lipschitz functions, also in the L2-topology.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 6; 2024: 1; 2025: 2; 2026: 1.

19 papers.

Sort:

  1. 2022Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networkscited by 26
  2. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  3. 2021An L2 Analysis of Reinforcement Learning in High Dimensions with Kernel and Neural Network Approximationcited by 13
  4. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  5. 2021Generalization Error of GAN from the Discriminator's Perspectivecited by 8
  6. 2023A Quantitative Functional Central Limit Theorem for Shallow Neural Networkscited by 8
  7. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  8. 2022A Mathematical Framework for Learning Probability Distributionscited by 7
  9. 2021A Class of Dimension-free Metrics for the Convergence of Empirical Measurescited by 5
  10. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 5
  11. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  12. 2023Reinforcement Learning with Function Approximation: From Linear to Nonlinearcited by 4
7 more
  1. 2023Embeddings between Barron spaces with higher order activation functionscited by 4
  2. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  3. 2022Qualitative neural network approximation over R and C: Elementary proofs for analytic and polynomial activationcited by 2
  4. 2023Delay-SDE-net: A deep learning approach for time series modelling with memory and uncertainty estimatescited by 2
  5. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  6. 2026Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Methodcited by 1
  7. 2021Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networkscited by 0