mlsanityData to 14 Oct 2026

On the growth of the parameters of approximating ReLU neural networks

arXiv 2406.14936 ↗First posted 21 Jun 2024cs.LG cs.NA math.NAcited by 0

Not on the map. Why?

Abstract (arXiv)

This work focuses on the analysis of fully connected feed forward ReLU neural networks as they approximate a given, smooth function. In contrast to conventionally studied universal approximation properties under increasing architectures, e.g., in terms of width or depth of the networks, we are concerned with the asymptotic growth of the parameters of approximating networks. Such results are of interest, e.g., for error analysis or consistency results for neural network training. The main result of our work is that, for a ReLU architecture with state of the art approximation error, the realizing parameters grow at most polynomially. The obtained rate with respect to a normalized network size is compared to existing results and is shown to be superior in most cases, in particular for high dimensional input.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

12 of 16 references.

References · 12

  1. 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 179
  2. 2020Deep Network Approximation for Smooth Functionscited by 164
  3. 2019Deep Network Approximation Characterized by Number of Neuronscited by 122
  4. 2019Error bounds for approximations with deep ReLU neural networks in Ws,p normscited by 95
  5. 2019Deep Neural Network Approximation Theorycited by 93
  6. 2021On the approximation of functions by tanh neural networkscited by 81
  7. 2019Approximation spaces of deep neural networkscited by 51
  8. 2019Nonlinear Approximation via Compositionscited by 38
  9. 2022Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activationscited by 29
  10. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  11. 2020Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisationcited by 7
  12. 2020Strong overall error analysis for the training of artificial neural networks via random initializationscited by 4

Built on by

Not cited yet.