mlsanityData to 7 Oct 2026

Complexity Measures for Neural Networks with General Activation Functions Using Path-based Norms

arXiv 2009.06132 ↗First posted 14 Sep 2020cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

A simple approach is proposed to obtain complexity controls for neural networks with general activation functions. The approach is motivated by approximating the general activation functions with one-dimensional ReLU networks, which reduces the problem to the complexity controls of ReLU networks. Specifically, we consider two-layer networks and deep residual networks, for which path-based norms are derived to control complexities. We also provide preliminary analyses of the function spaces induced by these norms and a priori estimates of the corresponding regularized estimators.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 7; 2024: 3; 2025: 2; 2026: 2.

19 papers.

Sort:

  1. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  2. 2024Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscapecited by 43
  3. 2023Penalising the biases in norm regularisation enforces sparsitycited by 17
  4. 2022Generalization Error Bounds for Deep Neural Networks Trained by SGDcited by 14
  5. 2022Approximation of Functionals by Neural Network without Curse of Dimensionalitycited by 8
  6. 2022Approximation results for Gradient Descent trained Shallow Neural Networks in 1dcited by 7
  7. 2023Space-Time Approximation with Shallow Neural Networks in Fourier Lebesgue spacescited by 5
  8. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  9. 2023Embeddings between Barron spaces with higher order activation functionscited by 4
  10. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  11. 2023Delay-SDE-net: A deep learning approach for time series modelling with memory and uncertainty estimatescited by 2
  12. 2023Embedding Inequalities for Barron-type Spacescited by 2
7 more
  1. 2023Approximation Results for Gradient Descent trained Neural Networkscited by 2
  2. 2024Solving the Poisson Equation with Dirichlet data by shallow ReLUα-networks: A regularity and approximation perspectivecited by 2
  3. 2024Approximation and Gradient Descent Training with Neural Networkscited by 1
  4. 2025Curse of Dimensionality in Neural Network Optimizationcited by 1
  5. 2025Statistical Learning Guarantees for Group-Invariant Barron Functionscited by 1
  6. 2026Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Methodcited by 1
  7. 2026Towards Initialization-dependent and Non-vacuous Generalization Bounds for Overparameterized Shallow Neural Networkscited by 0