Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes
Not on the map. Why?
Abstract (arXiv)
In this paper, we prove the universal consistency of wide and deep ReLU neural network classifiers trained on the logistic loss. We also give sufficient conditions for a class of probability measures for which classifiers based on neural networks achieve minimax optimal rates of convergence. The result applies to a wide range of known function classes. In particular, while most previous works impose explicit smoothness assumptions on the regression function, our framework encompasses more general settings. The proposed neural networks are either the minimizers of the logistic loss or the 0-1 loss. In the former case, they are interpolating classifiers that exhibit a benign overfitting behavior.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 10 of its 35 references that are in the map. First posted on arXiv 8 Jan 2024.
Builds on
10 of 35 references.
References · 10
- 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 306
- 2017Optimal approximation of piecewise smooth functions using deep ReLU neural networkscited by 178
- 2019Deep Neural Network Approximation Theorycited by 93
- 2019The phase diagram of approximation rates for deep neural networkscited by 86
- 2022Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Datacited by 74
- 2018Fast convergence rates of deep neural networks for classificationcited by 46
- 2021Convergence rates of deep ReLU networks for multiclass classificationcited by 21
- 2023Classification with Deep Neural Networks and Logistic Losscited by 16
- 2022Wide and Deep Neural Networks Achieve Optimality for Classificationcited by 12
- 2020Phase Transitions in Rate Distortion Theory and Deep Learningcited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.