Neural networks: deep, shallow, or in between?
Not on the map. Why?
Abstract (arXiv)
We give estimates from below for the error of approximation of a compact subset from a Banach space by the outputs of feed-forward neural networks with width W, depth l and Lipschitz activation functions. We show that, modulo logarithmic factors, rates better that entropy numbers' rates are possibly attainable only for neural networks for which the depth l goes to infinity, and that there is no gain if we fix the depth and let the width W go to infinity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 22 references that are in the map. First posted on arXiv 11 Oct 2023.
Builds on
5 of 22 references.
References · 5
- 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
- 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
- 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 51
- 2022A general approximation lower bound in Lp norm, with applications to feed-forward neural networkscited by 12
- 2022Limitations on approximation by deep and shallow neural networkscited by 4
Built on by
Not cited yet.