Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions
Not on the map. Why?
Abstract (arXiv)
Convolutional neural networks have shown impressive abilities in many applications, especially those related to the classification tasks. However, for the regression problem, the abilities of convolutional structures have not been fully understood, and further investigation is needed. In this paper, we consider the mean squared error analysis for deep convolutional neural networks. We show that, for additive ridge functions, convolutional neural networks followed by one fully connected layer with ReLU activation functions can reach optimal mini-max rates (up to a log factor). The input dimension only appears in the constant of convergence rates. This work shows the statistical optimality of convolutional neural networks and may shed light on why convolutional neural networks are able to behave well for high dimensional input.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 24 Feb 2022.
Builds on
24 of 45 references.
References · 24
- 2017Attention Is All You Needcited by 59,932
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Deep Learning in Neural Networks: An Overviewcited by 765
- 2016Error bounds for approximations with deep ReLU networkscited by 567
- 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
- 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
- 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 176
- 2018Universality of Deep Convolutional Neural Networkscited by 176
- 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
- 2018Deep Neural Networks Learn Non-Smooth Functions Effectivelycited by 57
- 2018Fast convergence rates of deep neural networks for classificationcited by 46
12 more
- 2020Towards Learning Convolutions from Scratchcited by 46
- 2019Approximation and Non-parametric Estimation of ResNet-type Convolutional Neural Networkscited by 40
- 2018Equivalence of approximation by convolutional neural networks and fully-connected networkscited by 38
- 2016Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networkscited by 26
- 2020Theory of Deep Convolutional Neural Networks II: Spherical Analysiscited by 22
- 2021Theory of Deep Convolutional Neural Networks III: Approximating Radial Functionscited by 22
- 2021Convergence rates of deep ReLU networks for multiclass classificationcited by 21
- 2019Finding the Needle in the Haystack with Convolutions: on the benefits of architectural biascited by 19
- 2021Universal Consistency of Deep Convolutional Neural Networkscited by 16
- 2021Approximation Properties of Deep ReLU CNNscited by 15
- 2020Sharp Rate of Convergence for Deep Neural Network Classifiers under the Teacher-Student Settingcited by 11
- 2017Minimax Lower Bounds for Ridge Combinations Including Neural Netscited by 4
Built on by
Citing papers per year: 2025: 1.
1 paper.