mlsanityData to 7 Oct 2026

Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

arXiv 2106.15002 ↗First posted 28 Jun 2021stat.ML cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

We study the variation space corresponding to a dictionary of functions in L2(Omega) for a bounded domain Omegasubset ℝd. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 9 of its 42 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

9 of 42 references.

References · 9

  1. 2019A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Casecited by 113
  2. 2019How do infinite width bounded norm networks look in function space?cited by 105
  3. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  4. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  5. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  6. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  7. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
  8. 2020Complexity Measures for Neural Networks with General Activation Functions Using Path-based Normscited by 19
  9. 2019Neural network integral representations with the ReLU activation functioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 3; 2023: 14; 2024: 8; 2025: 6; 2026: 14.

48 papers.

Sort:

  1. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 40
  2. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
  3. 2023Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compressioncited by 17
  4. 2021Approximation Properties of Deep ReLU CNNscited by 15
  5. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  6. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  7. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  8. 2024On the rates of convergence for learning with convolutional neural networkscited by 10
  9. 2021Stationary Density Estimation of Ito Diffusions Using Deep Learningcited by 9
  10. 2023ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Modelscited by 7
  11. 2023Weighted variation spaces and approximation by shallow ReLU networkscited by 7
  12. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6
36 more
  1. 2022Intrinsic dimensionality and generalization properties of the R-norm inductive biascited by 5
  2. 2023Function-Space Optimality of Neural Architectures with Multivariate Nonlinearitiescited by 5
  3. 2023Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spacescited by 5
  4. 2024ReLUs Are Sufficient for Learning Implicit Neural Representationscited by 5
  5. 2024Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domainscited by 5
  6. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  7. 2024Hypothesis Spaces for Deep Learningcited by 4
  8. 2023Transformers Can Solve Non-Linear and Non-Markovian Filtering Problems in Continuous Time For Conditionally Gaussian Signalscited by 3
  9. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  10. 2023Embedding Inequalities for Barron-type Spacescited by 2
  11. 2023Sharp Convergence Rates for Matching Pursuitcited by 2
  12. 2024Nonuniform random feature models using derivative informationcited by 2
  13. 2025Generalization Below the Edge of Stability: The Role of Data Geometrycited by 2
  14. 2025Time-Frequency Analysis for Neural Networkscited by 2
  15. 2026Shallow ReLUs Networks in Lp-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalizationcited by 2
  16. 2023Barron Space for Graph Convolution Neural Networkscited by 1
  17. 2025Barron Space Representations for Elliptic PDEs with Homogeneous Boundary Conditionscited by 1
  18. 2025Approximation Rates of Shallow Neural Networks: Barron Spaces, Activation Functions and Optimality Analysiscited by 1
  19. 2025Upper Approximation Bounds for Neural Oscillatorscited by 1
  20. 2026Regularity of Second-Order Elliptic PDEs in Spectral Barron Spacescited by 1
  21. 2026Does Sparse Connectivity Improve Generalization? Convolutional Networks Below the Edge of Stabilitycited by 1
  22. 2026Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Methodcited by 1
  23. 2022Optimal Learningcited by 0
  24. 2024Dimension-independent learning rates for high-dimensional classification problemscited by 0
  25. 2024Decomposition of one-layer neural networks via the infinite sum of reproducing kernel Banach spacescited by 0
  26. 2025Path Regularization: A Near-Complete and Optimal Nonasymptotic Generalization Theory for Multilayer Neural Networks and Double Descent Phenomenoncited by 0
  27. 2026Sparse RBF Networks for PDEs and nonlocal equations: function space theory, operator calculus, and training algorithmscited by 0
  28. 2026Certified and accurate computation of function space norms of deep neural networkscited by 0
  29. 2026The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraintscited by 0
  30. 2026Do Neural Networks Really Beat the Curse of Dimensionality? A Bit-Complexity Viewcited by 0
  31. 2026Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samplescited by 0
  32. 2026Approximation Rates for Metaplectic Neural Networkscited by 0
  33. 2026Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spacescited by 0
  34. 2026Shallow neural network approximation in mixed Sobolev spacescited by 0
  35. 2026Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networkscited by 0
  36. 2026Minimax rates for learning spectral Barron functions by deep ReLU neural networkscited by 0