# Convergence rates of deep ReLU networks for multiclass classification

arXiv: https://arxiv.org/abs/2108.00969 · HTML: https://mlsanity.com/paper/2108.00969/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 37 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 7 of 7)

- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018)
- [On the rate of convergence of fully connected very deep neural network regression estimates](https://mlsanity.com/paper/1908.11133/) (2019)
- [On the rate of convergence of image classifiers based on convolutional neural networks](https://mlsanity.com/paper/2003.01526/) (2020)
- [Non-asymptotic Excess Risk Bounds for Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/2105.00292/) (2021)

## Built on by (most cited first, 21 of 21)

- [Adaptive novelty detection with false discovery rate guarantee](https://mlsanity.com/paper/2208.06685/) (2022)
- [Automatic Change-Point Detection in Time Series via Deep Learning](https://mlsanity.com/paper/2211.03860/) (2022)
- [On the rates of convergence for learning with convolutional neural networks](https://mlsanity.com/paper/2403.16459/) (2024)
- [Misclassification bounds for PAC-Bayesian sparse deep learning](https://mlsanity.com/paper/2405.01304/) (2024)
- [Deep Neural Network Classifier for Multi-dimensional Functional Data](https://mlsanity.com/paper/2205.08592/) (2022)
- [How many samples are needed to train a deep neural network?](https://mlsanity.com/paper/2405.16696/) (2024)
- [Analysis of convolutional neural network image classifiers in a rotationally symmetric model](https://mlsanity.com/paper/2205.05500/) (2022)
- [Classification of Data Generated by Gaussian Mixture Models Using Deep ReLU Networks](https://mlsanity.com/paper/2308.08030/) (2023)
- [Deep learning with missing data](https://mlsanity.com/paper/2504.15388/) (2025)
- [Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs](https://mlsanity.com/paper/2505.17599/) (2025)
- [Sharper Generalization Bounds for Transformer](https://mlsanity.com/paper/2603.21541/) (2026)
- [Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions](https://mlsanity.com/paper/2202.12119/) (2022)
- [Multiclass classification for multidimensional functional data through deep neural networks](https://mlsanity.com/paper/2305.13349/) (2023)
- [Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes](https://mlsanity.com/paper/2401.04286/) (2024)
- [Orthogonal Series Estimation for the Ratio of Conditional Expectation Functions](https://mlsanity.com/paper/2212.13145/) (2022)
- [Consistency for Large Neural Networks: Regression and Classification](https://mlsanity.com/paper/2409.14123/) (2024)
- [A Two-Sample Test of Text Generation Similarity](https://mlsanity.com/paper/2505.05269/) (2025)
- [Super-fast Rates of Convergence for Neural Network Classifiers under the Hard Margin Condition](https://mlsanity.com/paper/2505.08262/) (2025)
- [Plug-In Classification of Drift Functions in Diffusion Processes Using Neural Networks](https://mlsanity.com/paper/2602.02791/) (2026)
- [Adversarial Robustness of NTK Neural Networks](https://mlsanity.com/paper/2604.25965/) (2026)
- [Rate-optimal neural boundary detection from unlabeled noisy images](https://mlsanity.com/paper/2606.00715/) (2026)

## Abstract (arXiv)

For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.