# Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds

arXiv: https://arxiv.org/abs/2610.02253 · HTML: https://mlsanity.com/paper/2610.02253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 8 of 8)

- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression](https://mlsanity.com/paper/2410.06378/) (2024)
- [Almost Sure Convergence of Dropout Algorithms for Neural Networks](https://mlsanity.com/paper/2002.02247/) (2020)
- [When Neurons Fail](https://mlsanity.com/paper/1706.08884/) (2017)
- [Universal Approximation in Dropout Neural Networks](https://mlsanity.com/paper/2012.10351/) (2020)
- [Information contraction in noisy binary neural networks and its implications](https://mlsanity.com/paper/2101.11750/) (2021)

## Abstract (arXiv)

The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realization. In this work, we study approximation of the unit ball of Wn,∞([0,1]d) by ReLU networks whose edges are retained independently with probability p. The approximation error is measured uniformly over the input domain, and the guarantee holds with probability at least 1-δ for a single sampled network. We construct networks of constant depth and size widetilde On,d(p-9varepsilon^-maxd/n,2 log(1/δ)). The construction combines bounded local subnetworks, localization on a successful approximation event, and a multiscale Taylor decomposition. Conversely, Sobolev capacity imposes a lower bound on the number of surviving edges, while approximation of a fixed affine function requires an output-layer cost of order ((1-p)/p)varepsilon-2log(1/δ) at sufficiently high confidence. For fixed p∈(0,1) and δ<min1/2,1-p, the upper and lower bounds match in the accuracy exponent under a fixed or logarithmic depth budget. When d≤2n, they also match in confidence up to logarithms of accuracy. We extend the lower bounds to Wn,r targets with Ls error, and distinguish this extension from the upper bound for Wn,∞. The optimal retention dependence and logarithmic factors remain open.