mlsanityData to 7 Oct 2026

Embeddings between Barron spaces with higher order activation functions

arXiv 2305.15839 ↗First posted 25 May 2023stat.ML cs.LG math.FAcited by 4

Not on the map. Why?

Abstract (arXiv)

The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing push-forward maps on the measures μ used to represent functions f. An activation function of particular interest is the rectified power unit (operatornameRePU) given by operatornameRePUs(x)=max(0,x)s. For many commonly used activation functions, the well-known Taylor remainder theorem can be used to construct a push-forward map, which allows us to prove the embedding of the associated Barron space into a Barron space with a operatornameRePU as activation function. Moreover, the Barron spaces associated with the operatornameRePUs have a hierarchical structure similar to the Sobolev spaces Hm.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 14 of its 29 references that are in the map. First posted on arXiv 25 May 2023.

Builds on

14 of 29 references.

References · 14

  1. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  2. 2019Searching for MobileNetV3cited by 1,984
  3. 2020Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don'tcited by 94
  4. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  5. 2015Representation Benefits of Deep Feedforward Networkscited by 84
  6. 2020Representation formulas and pointwise properties for Barron functionscited by 62
  7. 2020Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functionscited by 50
  8. 2021Understanding neural networks with reproducing kernel Banach spacescited by 50
  9. 2020Neural network approximation and estimation of classifiers with classification boundary in a Barron classcited by 28
  10. 2020Kolmogorov Width Decay and Poor Approximators in Machine Learning: Shallow Neural Networks, Random Feature Models and Neural Tangent Kernelscited by 19
  11. 2020Complexity Measures for Neural Networks with General Activation Functions Using Path-based Normscited by 19
  12. 2020Some observations on high-dimensional partial differential equations with Barron datacited by 16
2 more
  1. 2022Duality for Neural Networks through Reproducing Kernel Banach Spacescited by 15
  2. 2021Growing Cosine Unit: A Novel Oscillatory Activation Function That Can Speedup Training and Reduce Parameters in Convolutional Neural Networkscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2024A Lipschitz spaces view of infinitely wide shallow neural networkscited by 2
  2. 2023Learning a Sparse Representation of Barron Functions with the Inverse Scale Space Flowcited by 1
  3. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  4. 2026A Dual Certificate Approach to Sparsity in Infinite-Width Shallow Neural Networkscited by 0