# Characterization of the Variation Spaces Corresponding to Shallow Neural Networks

arXiv: https://arxiv.org/abs/2106.15002 · HTML: https://mlsanity.com/paper/2106.15002/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [How do infinite width bounded norm networks look in function space?](https://mlsanity.com/paper/1902.05040/) (2019)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [Representation formulas and pointwise properties for Barron functions](https://mlsanity.com/paper/2006.05982/) (2020)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [The Barron Space and the Flow-induced Function Spaces for Neural Network Models](https://mlsanity.com/paper/1906.08039/) (2019)
- [Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms](https://mlsanity.com/paper/2009.06132/) (2020)
- [Neural network integral representations with the ReLU activation function](https://mlsanity.com/paper/1910.02743/) (2019)

## Built on by (most cited first, 30 of 48)

- [Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks](https://mlsanity.com/paper/2109.08844/) (2021)
- [Nonparametric regression using over-parameterized shallow ReLU neural networks](https://mlsanity.com/paper/2306.08321/) (2023)
- [Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression](https://mlsanity.com/paper/2305.16534/) (2023)
- [Approximation Properties of Deep ReLU CNNs](https://mlsanity.com/paper/2109.00190/) (2021)
- [Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regression](https://mlsanity.com/paper/2304.01561/) (2023)
- [On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theorem](https://mlsanity.com/paper/2308.05509/) (2023)
- [Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networks](https://mlsanity.com/paper/2307.15285/) (2023)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Stationary Density Estimation of Ito Diffusions Using Deep Learning](https://mlsanity.com/paper/2109.03992/) (2021)
- [ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models](https://mlsanity.com/paper/2305.15598/) (2023)
- [Weighted variation spaces and approximation by shallow ReLU networks](https://mlsanity.com/paper/2307.15772/) (2023)
- [Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transform](https://mlsanity.com/paper/2408.10996/) (2024)
- [Intrinsic dimensionality and generalization properties of the R-norm inductive bias](https://mlsanity.com/paper/2206.05317/) (2022)
- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spaces](https://mlsanity.com/paper/2312.08461/) (2023)
- [ReLUs Are Sufficient for Learning Implicit Neural Representations](https://mlsanity.com/paper/2406.02529/) (2024)
- [Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains](https://mlsanity.com/paper/2411.04108/) (2024)
- [Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionality](https://mlsanity.com/paper/2209.01173/) (2022)
- [Transformers Can Solve Non-Linear and Non-Markovian Filtering Problems in Continuous Time For Conditionally Gaussian Signals](https://mlsanity.com/paper/2310.19603/) (2023)
- [Hypothesis Spaces for Deep Learning](https://mlsanity.com/paper/2403.03353/) (2024)
- [Minimum norm interpolation by perceptra: Explicit regularization and implicit bias](https://mlsanity.com/paper/2311.06138/) (2023)
- [Generalization Below the Edge of Stability: The Role of Data Geometry](https://mlsanity.com/paper/2510.18120/) (2025)
- [Embedding Inequalities for Barron-type Spaces](https://mlsanity.com/paper/2305.19082/) (2023)
- [Sharp Convergence Rates for Matching Pursuit](https://mlsanity.com/paper/2307.07679/) (2023)
- [Nonuniform random feature models using derivative information](https://mlsanity.com/paper/2410.02132/) (2024)
- [Time-Frequency Analysis for Neural Networks](https://mlsanity.com/paper/2512.15992/) (2025)
- [Shallow ReLUs Networks in Lp-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization](https://mlsanity.com/paper/2605.18468/) (2026)
- [Barron Space for Graph Convolution Neural Networks](https://mlsanity.com/paper/2311.02838/) (2023)
- [Barron Space Representations for Elliptic PDEs with Homogeneous Boundary Conditions](https://mlsanity.com/paper/2508.07559/) (2025)
- [Approximation Rates of Shallow Neural Networks: Barron Spaces, Activation Functions and Optimality Analysis](https://mlsanity.com/paper/2510.18388/) (2025)

## Abstract (arXiv)

We study the variation space corresponding to a dictionary of functions in L2(Omega) for a bounded domain Omegasubset ℝd. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.