mlsanityData to 7 Oct 2026

Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controls

arXiv 1607.07819 ↗First posted 26 Jul 2016stat.ML math.ST stat.THcited by 84

Not on the map. Why?

Abstract (arXiv)

We establish L∞ and L2 error bounds for functions of many variables that are approximated by linear combinations of ReLU (rectified linear unit) and squared ReLU ridge functions with ℓ1 and ℓ0 controls on their inner and outer parameters. With the squared ReLU ridge function, we show that the L2 approximation error is inversely proportional to the inner layer ℓ0 sparsity and it need only be sublinear in the outer layer ℓ0 sparsity. Our constructions are obtained using a variant of the Jones-Barron probabilistic method, which can be interpreted as either stratified sampling with proportionate allocation or two-stage cluster sampling. We also provide companion error lower bounds that reveal near optimality of our constructions. Despite the sparsity assumptions, we showcase the richness and flexibility of these ridge combinations by defining a large family of functions, in terms of certain spectral conditions, that are particularly well approximated by them.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 13; 2023: 21; 2024: 12; 2025: 10; 2026: 10.

84 papers.

Sort:

By area:

  1. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  2. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 50
  3. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  4. 2021Near-Minimax Optimal Estimation With Shallow ReLU Neural Networkscited by 40
  5. 2024On the expressiveness and spectral bias of KANscited by 36
  6. 2021Random feature neural networks learn Black-Scholes type PDEs without curse of dimensionalitycited by 34
  7. 2021On the Approximation Power of Two-Layer Networks of Random ReLUscited by 29
  8. 2022Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activationscited by 29
  9. 2023Approximation Rate of the Transformer Architecture for Sequence Modelingcited by 28
  10. 2021On the Representation of Solutions to Elliptic PDEs in Barron Spacescited by 25
  11. 2023Energy-guided Entropic Neural Optimal Transportcited by 25
  12. 2023Nonparametric regression using over-parameterized shallow ReLU neural networkscited by 24
38 more
  1. 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
  2. 2021A Priori Generalization Error Analysis of Two-Layer Neural Networks for Solving High Dimensional Schr"odinger Eigenvalue Problemscited by 19
  3. 2021A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equationscited by 18
  4. 2021Depth separation beyond radial functionscited by 18
  5. 2021Approximation Properties of Deep ReLU CNNscited by 15
  6. 2023Optimal rates of approximation by shallow ReLUk neural networks and applications to nonparametric regressioncited by 15
  7. 2023The Expressive Power of Tuning Only the Normalization Layerscited by 13
  8. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theoremcited by 13
  9. 2021ReLU Deep Neural Networks from the Hierarchical Basis Perspectivecited by 12
  10. 2022Active Learning for Single Neuron Models with Lipschitz Non-Linearitiescited by 12
  11. 2023Rates of Approximation by ReLU Shallow Neural Networkscited by 12
  12. 2024Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributionscited by 12
  13. 2023Optimal Approximation of Zonoids and Uniform Approximation by Shallow Neural Networkscited by 11
  14. 2023Max-Sliced Mutual Informationcited by 10
  15. 2021Scaling Up Bayesian Uncertainty Quantification for Inverse Problems using Deep Neural Networkscited by 9
  16. 2021Optimal learning of high-dimensional classification problems using deep neural networkscited by 9
  17. 2022Learning Ability of Interpolating Deep Convolutional Neural Networkscited by 9
  18. 2022Approximation of Functionals by Neural Network without Curse of Dimensionalitycited by 8
  19. 2023Approximation of Nonlinear Functionals Using Deep ReLU Networkscited by 8
  20. 2023Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regressioncited by 8
  21. 2023Expressivity and Approximation Properties of Deep Neural Networks with ReLUk Activationcited by 8
  22. 2024Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networkscited by 8
  23. 2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
  24. 2023Weighted variation spaces and approximation by shallow ReLU networkscited by 7
  25. 2022Lp sampling numbers for the Fourier-analytic Barron spacecited by 6
  26. 2022Approximation analysis of CNNs from a feature extraction viewcited by 6
  27. 2024Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializationscited by 6
  28. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6
  29. 2022Universality and approximation bounds for echo state networks with random weightscited by 5
  30. 2023Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spacescited by 5
  31. 2024ReLUs Are Sufficient for Learning Implicit Neural Representationscited by 5
  32. 2025Deep Neural Networks with General Activations: Super-Convergence in Sobolev Normscited by 5
  33. 2021Finding Everything within Random Binary Networkscited by 4
  34. 2021A Unified and Constructive Framework for the Universality of Neural Networkscited by 4
  35. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  36. 2022SignReLU neural network and its approximation abilitycited by 4
  37. 2023Tractability of approximation by general shallow networkscited by 4
  38. 2021Tighter Sparse Approximation Bounds for ReLU Neural Networkscited by 3

50 most cited of 84 All 84 (JSON)