# Concentration inequalities and optimal number of layers for stochastic deep neural networks

arXiv: https://arxiv.org/abs/2206.11241 · HTML: https://mlsanity.com/paper/2206.11241/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 22 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Users](https://mlsanity.com/paper/2007.06823/) (2020)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Bayesian Neural Networks: An Introduction and Survey](https://mlsanity.com/paper/2006.12024/) (2020)
- [A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm](https://mlsanity.com/paper/1902.03736/) (2019)
- [Tropical Geometry of Deep Neural Networks](https://mlsanity.com/paper/1805.07091/) (2018)
- [Neural Network Approximation: Three Hidden Layers Are Enough](https://mlsanity.com/paper/2010.14075/) (2020)
- [Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-offs by Selective Execution](https://mlsanity.com/paper/1701.00299/) (2017)
- [On the Decision Boundaries of Neural Networks: A Tropical Geometry Perspective](https://mlsanity.com/paper/2002.08838/) (2020)
- [Stochastic Deep Networks](https://mlsanity.com/paper/1811.07429/) (2018)
- [Stochastic Feedforward Neural Networks: Universal Approximation](https://mlsanity.com/paper/1910.09763/) (2019)
- [Stochastic Neural Network with Kronecker Flow](https://mlsanity.com/paper/1906.04282/) (2019)
- [Intelligence plays dice: Stochasticity is essential for machine learning](https://mlsanity.com/paper/2008.07496/) (2020)
- [Simplified Stochastic Feedforward Neural Networks](https://mlsanity.com/paper/1704.03188/) (2017)
- [Deviation bound for non-causal machine learning](https://mlsanity.com/paper/2009.08905/) (2020)

## Built on by (most cited first, 1 of 1)

- [EWasteNet: A Two-Stream Data Efficient Image Transformer Approach for E-Waste Classification](https://mlsanity.com/paper/2311.12823/) (2023)

## Abstract (arXiv)

We state concentration inequalities for the output of the hidden layers of a stochastic deep neural network (SDNN), as well as for the output of the whole SDNN. These results allow us to introduce an expected classifier (EC), and to give probabilistic upper bound for the classification error of the EC. We also state the optimal number of layers for the SDNN via an optimal stopping procedure. We apply our analysis to a stochastic version of a feedforward neural network with ReLU activation function.