# Dropout as data augmentation

arXiv: https://arxiv.org/abs/1506.08700 · HTML: https://mlsanity.com/paper/1506.08700/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2015.

## Built on by (most cited first, 27 of 27)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders](https://mlsanity.com/paper/2104.08027/) (2021)
- [Reflash Dropout in Image Super-Resolution](https://mlsanity.com/paper/2112.12089/) (2021)
- [Mine Your Own vieW: Self-Supervised Learning Through Across-Sample Prediction](https://mlsanity.com/paper/2102.10106/) (2021)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [The good, the bad and the ugly sides of data augmentation: An implicit spectral regularization perspective](https://mlsanity.com/paper/2210.05021/) (2022)
- [SimCSE++: Improving Contrastive Learning for Sentence Embeddings from Two Perspectives](https://mlsanity.com/paper/2305.13192/) (2023)
- [A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments](https://mlsanity.com/paper/2202.08325/) (2022)
- [Rethinking Data Distillation: Do Not Overlook Calibration](https://mlsanity.com/paper/2307.12463/) (2023)
- [RepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classification](https://mlsanity.com/paper/2405.02996/) (2024)
- [Tackling covariate shift with node-based Bayesian neural networks](https://mlsanity.com/paper/2206.02435/) (2022)
- [CoReFace: Sample-Guided Contrastive Regularization for Deep Face Recognition](https://mlsanity.com/paper/2304.11668/) (2023)
- [Anchor Data Augmentation](https://mlsanity.com/paper/2311.06965/) (2023)
- [A Survey on Dropout Methods and Experimental Verification in Recommendation](https://mlsanity.com/paper/2204.02027/) (2022)
- [ActNetFormer: Transformer-ResNet Hybrid Method for Semi-Supervised Action Recognition in Videos](https://mlsanity.com/paper/2404.06243/) (2024)
- [Frequency Dropout: Feature-Level Regularization via Randomized Filtering](https://mlsanity.com/paper/2209.09844/) (2022)
- [Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach](https://mlsanity.com/paper/2501.01132/) (2025)
- [Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction](https://mlsanity.com/paper/2206.00913/) (2022)
- [Deep Augmentation: Dropout as Augmentation for Self-Supervised Learning](https://mlsanity.com/paper/2303.14537/) (2023)
- [Weights Augmentation: it has never ever ever ever let her model down](https://mlsanity.com/paper/2405.19590/) (2024)
- [SkIn: Skimming-Intensive Long-Text Classification Using BERT for Medical Corpus](https://mlsanity.com/paper/2209.05741/) (2022)
- [Online Non-Destructive Moisture Content Estimation of Filter Media During Drying Using Artificial Neural Networks](https://mlsanity.com/paper/2303.15570/) (2023)
- [ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States](https://mlsanity.com/paper/2305.18684/) (2023)
- [ApproBiVT: Lead ASR Models to Generalize Better Using Approximated Bias-Variance Tradeoff Guided Early Stopping and Checkpoint Averaging](https://mlsanity.com/paper/2308.02870/) (2023)
- [Relevance-driven Input Dropout: an Explanation-guided Regularization Technique](https://mlsanity.com/paper/2505.21595/) (2025)
- [A Neural Network-Based Real-time Casing Collar Recognition System for Downhole Instruments](https://mlsanity.com/paper/2512.22901/) (2025)
- [Towards a Data-Parameter Correspondence for LLMs: A Preliminary Discussion](https://mlsanity.com/paper/2604.17384/) (2026)

## Abstract (arXiv)

Dropout is typically interpreted as bagging a large number of models sharing parameters. We show that using dropout in a network can also be interpreted as a kind of data augmentation in the input space without domain knowledge. We present an approach to projecting the dropout noise within a network back into the input space, thereby generating augmented versions of the training data, and we show that training a deterministic network on the augmented samples yields similar results. Finally, we propose a new dropout noise scheme based on our observations and show that it improves dropout results without adding significant computational cost.