mlsanityData to 14 Oct 2026

Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks

arXiv 2101.12365 ↗First posted 29 Jan 2021stat.ML cs.IT cs.LGcited by 86

Not on the map. Why?

Abstract (arXiv)

In this article, we study approximation properties of the variation spaces corresponding to shallow neural networks with a variety of activation functions. We introduce two main tools for estimating the metric entropy, approximation rates, and n-widths of these spaces. First, we introduce the notion of a smoothly parameterized dictionary and give upper bounds on the non-linear approximation rates, metric entropy and n-widths of their absolute convex hull. The upper bounds depend upon the order of smoothness of the parameterization. This result is applied to dictionaries of ridge functions corresponding to shallow neural networks, and they improve upon existing results in many cases. Next, we provide a method for lower bounding the metric entropy and n-widths of variation spaces which contain certain classes of ridge functions. This result gives sharp lower bounds on the L2-approximation rates, metric entropy, and n-widths for variation spaces corresponding to neural networks with a range of important activation functions, including ReLUk activation functions and sigmoidal activation functions with bounded variation.

Key facts. Cited by 86 papers in the map (data through 14 Oct 2026). Builds on 5 of its 58 references that are in the map. First posted on arXiv 29 Jan 2021.

Builds on

5 of 58 references.

References · 5

  1. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 331
  2. 2019A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Casecited by 114
  3. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  4. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  5. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 20; 2024: 11; 2025: 15; 2026: 24.

86 papers.

Sort:

By area:

  1. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 51
  2. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  3. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  4. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 41
  5. 2024On the expressiveness and spectral bias of KANscited by 37
  6. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  7. 2023Deep Learning Meets Sparse Regularization: A Signal Processing Perspectivecited by 21
  8. 2022Optimal transport map estimation in general function spacescited by 20
  9. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  10. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  11. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  12. 2022A general approximation lower bound in Lp norm, with applications to feed-forward neural networkscited by 12
38 more
  1. 2024Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributionscited by 12
  2. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  3. 2022DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learningcited by 10
  4. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  5. 2021Optimal learning of high-dimensional classification problems using deep neural networkscited by 9
  6. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 9
  7. 2023ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Modelscited by 8
  8. 2023Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applicationscited by 8
  9. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  10. 2023Weighted variation spaces and approximation by shallow ReLU networkscited by 7
  11. 2024Universal approximation results for neural networks with non-polynomial activation function over non-compact domainscited by 7
  12. 2022Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Spacecited by 6
  13. 2022Lp sampling numbers for the Fourier-analytic Barron spacecited by 6
  14. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6
  15. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 6
  16. 2023Function-Space Optimality of Neural Architectures with Multivariate Nonlinearitiescited by 5
  17. 2023Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spacescited by 5
  18. 2024Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domainscited by 5
  19. 2025Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspectivecited by 5
  20. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  21. 2022Limitations on approximation by deep and shallow neural networkscited by 4
  22. 2023Embeddings between Barron spaces with higher order activation functionscited by 4
  23. 2023Tractability of approximation by general shallow networkscited by 4
  24. 2023Optimal Deep Neural Network Approximation for Korobov Functions with respect to Sobolev Normscited by 4
  25. 2025Higher Order Approximation Rates for ReLU CNNs in Korobov Spacescited by 4
  26. 2026A short tour of operator learning theory: Convergence rates, statistical limits, and open questionscited by 4
  27. 2023Sharp Lower Bounds on Interpolation by Deep ReLU Neural Networks at Irregularly Spaced Datacited by 3
  28. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  29. 2025Orthogonal greedy algorithm for linear operator learning with shallow neural networkcited by 3
  30. 2025Some Super-approximation Rates of ReLU Neural Networks for Korobov Functionscited by 3
  31. 2026Learning Sparse Compositional Functions with Norm-Constrained Neural Networkscited by 3
  32. 2026Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networkscited by 3
  33. 2021Nonasymptotic theory for two-layer neural networks: Beyond the bias-variance trade-offcited by 2
  34. 2023Embedding Inequalities for Barron-type Spacescited by 2
  35. 2023Sharp Convergence Rates for Matching Pursuitcited by 2
  36. 2024Nonuniform random feature models using derivative informationcited by 2
  37. 2025Nonlocal techniques for the analysis of deep ReLU neural network approximationscited by 2
  38. 2025Time-Frequency Analysis for Neural Networkscited by 2

50 most cited of 86 All 86 (JSON)