# Weighted variation spaces and approximation by shallow ReLU networks

arXiv: https://arxiv.org/abs/2307.15772 · HTML: https://mlsanity.com/paper/2307.15772/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 46 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Path-SGD: Path-Normalized Optimization in Deep Neural Networks](https://mlsanity.com/paper/1506.02617/) (2015)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [Optimal Approximation Rate of ReLU Networks in terms of Width and Depth](https://mlsanity.com/paper/2103.00502/) (2021)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networks](https://mlsanity.com/paper/2101.12365/) (2021)
- [Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with  ℓ1  and  ℓ0  Controls](https://mlsanity.com/paper/1607.07819/) (2016)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Approximation spaces of deep neural networks](https://mlsanity.com/paper/1905.01208/) (2019)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Characterization of the Variation Spaces Corresponding to Shallow Neural Networks](https://mlsanity.com/paper/2106.15002/) (2021)
- [Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks](https://mlsanity.com/paper/2109.08844/) (2021)
- [Two-layer neural networks with values in a Banach space](https://mlsanity.com/paper/2105.02095/) (2021)
- [Sparsity of solutions for variational inverse problems with finite-dimensional data](https://mlsanity.com/paper/1809.05045/) (2018)
- [On Representer Theorems and Convex Regularization](https://mlsanity.com/paper/1806.09810/) (2018)
- [A unifying representer theorem for inverse problems and machine learning](https://mlsanity.com/paper/1903.00687/) (2019)
- [Deep Learning Meets Sparse Regularization: A Signal Processing Perspective](https://mlsanity.com/paper/2301.09554/) (2023)
- [Dimension independent bounds for general shallow networks](https://mlsanity.com/paper/1908.09880/) (2019)

## Built on by (most cited first, 7 of 7)

- [Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities](https://mlsanity.com/paper/2310.03696/) (2023)
- [Weighted Sobolev Approximation Rates for Neural Networks on Unbounded Domains](https://mlsanity.com/paper/2411.04108/) (2024)
- [Generalization Below the Edge of Stability: The Role of Data Geometry](https://mlsanity.com/paper/2510.18120/) (2025)
- [A Lipschitz spaces view of infinitely wide shallow neural networks](https://mlsanity.com/paper/2410.14591/) (2024)
- [Time-Frequency Analysis for Neural Networks](https://mlsanity.com/paper/2512.15992/) (2025)
- [Variation Spaces for Encoder--Decoder Neural Operators: Approximation and Generalization](https://mlsanity.com/paper/2606.01244/) (2026)
- [Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method](https://mlsanity.com/paper/2607.25100/) (2026)

## Abstract (arXiv)

We investigate the approximation of functions f on a bounded domain Omegasubset ℝd by the outputs of single-hidden-layer ReLU neural networks of width n. This form of nonlinear n-term dictionary approximation has been intensely studied since it is the simplest case of neural network approximation (NNA). There are several celebrated approximation results for this form of NNA that introduce novel model classes of functions on Omega whose approximation rates do not grow unbounded with the input dimension. These novel classes include Barron classes, and classes based on sparsity or variation such as the Radon-domain BV classes. The present paper is concerned with the definition of these novel model classes on domains Omega. The current definition of these model classes does not depend on the domain Omega. A new and more proper definition of model classes on domains is given by introducing the concept of weighted variation spaces. These new model classes are intrinsic to the domain itself. The importance of these new model classes is that they are strictly larger than the classical (domain-independent) classes. Yet, it is shown that they maintain the same NNA rates.