# Altitude Training: Strong Bounds for Single-Layer Dropout

arXiv: https://arxiv.org/abs/1407.3289 · HTML: https://mlsanity.com/paper/1407.3289/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2014.

## Built on by (most cited first, 2 of 2)

- [Implicit regularization of dropout](https://mlsanity.com/paper/2207.05952/) (2022)
- [Salient Objects in Clutter](https://mlsanity.com/paper/2105.03053/) (2021)

## Abstract (arXiv)

Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent in the generalization bound for empirical risk minimization. Dropout achieves this gain much like a marathon runner who practices at altitude: once a classifier learns to perform reasonably well on training examples that have been artificially corrupted by dropout, it will do very well on the uncorrupted test set. We also show that, under similar conditions, dropout preserves the Bayes decision boundary and should therefore induce minimal bias in high dimensions.