# Guided Dropout

arXiv: https://arxiv.org/abs/1812.03965 · HTML: https://mlsanity.com/paper/1812.03965/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2018.

## Built on by (most cited first, 14 of 14)

- [Codebook Features: Sparse and Discrete Interpretability for Neural Networks](https://mlsanity.com/paper/2310.17230/) (2023)
- [Split-and-Bridge: Adaptable Class Incremental Learning within a Single Neural Network](https://mlsanity.com/paper/2107.01349/) (2021)
- [Aux-Drop: Handling Haphazard Inputs in Online Learning Using Auxiliary Dropouts](https://mlsanity.com/paper/2303.05155/) (2023)
- [Image Classification with Small Datasets: Overview and Benchmark](https://mlsanity.com/paper/2212.12478/) (2022)
- [FocusedDropout for Convolutional Neural Network](https://mlsanity.com/paper/2103.15425/) (2021)
- [Information Geometry of Dropout Training](https://mlsanity.com/paper/2206.10936/) (2022)
- [More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing](https://mlsanity.com/paper/2410.08003/) (2024)
- [Exploring Dropout Discriminator for Domain Adaptation](https://mlsanity.com/paper/2107.04231/) (2021)
- [Feature Mining: A Novel Training Strategy for Convolutional Neural Network](https://mlsanity.com/paper/2107.08421/) (2021)
- [Improving the Robustness and Generalization of Deep Neural Network with Confidence Threshold Reduction](https://mlsanity.com/paper/2206.00913/) (2022)
- [No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets](https://mlsanity.com/paper/2309.01694/) (2023)
- [Deep Neural Networks Based Weight Approximation and Computation Reuse for 2-D Image Classification](https://mlsanity.com/paper/2105.02954/) (2021)
- [Feature Weaken: Vicinal Data Augmentation for Classification](https://mlsanity.com/paper/2211.10944/) (2022)
- [Y-Drop: A Conductance based Dropout for fully connected layers](https://mlsanity.com/paper/2409.09088/) (2024)

## Abstract (arXiv)

Dropout is often used in deep neural networks to prevent over-fitting. Conventionally, dropout training invokes textitrandom drop of nodes from the hidden layers of a Neural Network. It is our hypothesis that a guided selection of nodes for intelligent dropout can lead to better generalization as compared to the traditional dropout. In this research, we propose "guided dropout" for training deep neural network which drop nodes by measuring the strength of each node. We also demonstrate that conventional dropout is a specific case of the proposed guided dropout. Experimental evaluation on multiple datasets including MNIST, CIFAR10, CIFAR100, SVHN, and Tiny ImageNet demonstrate the efficacy of the proposed guided dropout.