# From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information

arXiv: https://arxiv.org/abs/1808.08149 · HTML: https://mlsanity.com/paper/1808.08149/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Aug 2018.

## Built on by (most cited first, 3 of 3)

- [Enhanced Aspect-Based Sentiment Analysis Models with Progressive Self-supervised Attention Learning](https://mlsanity.com/paper/2103.03446/) (2021)
- [Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data](https://mlsanity.com/paper/2512.23422/) (2025)
- [Perturbation-based Self-supervised Attention for Attention Bias in Text Classification](https://mlsanity.com/paper/2305.15684/) (2023)

## Abstract (arXiv)

Dropout is used to avoid overfitting by randomly dropping units from the neural networks during training. Inspired by dropout, this paper presents GI-Dropout, a novel dropout method integrating with global information to improve neural networks for text classification. Unlike the traditional dropout method in which the units are dropped randomly according to the same probability, we aim to use explicit instructions based on global information of the dataset to guide the training process. With GI-Dropout, the model is supposed to pay more attention to inapparent features or patterns. Experiments demonstrate the effectiveness of the dropout with global information on seven text classification tasks, including sentiment analysis and topic classification.