# Adversarial Dropout for Supervised and Semi-supervised Learning

arXiv: https://arxiv.org/abs/1707.03631 · HTML: https://mlsanity.com/paper/1707.03631/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2017.

## Built on by (most cited first, 30 of 39)

- [A Fourier-based Framework for Domain Generalization](https://mlsanity.com/paper/2105.11120/) (2021)
- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Adversarially Adaptive Normalization for Single Domain Generalization](https://mlsanity.com/paper/2106.01899/) (2021)
- [Deep Clustering: A Comprehensive Survey](https://mlsanity.com/paper/2210.04142/) (2022)
- [ABC: Auxiliary Balanced Classifier for Class-imbalanced Semi-supervised Learning](https://mlsanity.com/paper/2110.10368/) (2021)
- [Debiased Self-Training for Semi-Supervised Learning](https://mlsanity.com/paper/2202.07136/) (2022)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Cross-View Regularization for Domain Adaptive Panoptic Segmentation](https://mlsanity.com/paper/2103.02584/) (2021)
- [ConMatch: Semi-Supervised Learning with Confidence-Guided Consistency Regularization](https://mlsanity.com/paper/2208.08631/) (2022)
- [DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization](https://mlsanity.com/paper/2308.10285/) (2023)
- [Semi-supervised learning made simple with self-supervised clustering](https://mlsanity.com/paper/2306.07483/) (2023)
- [A soft nearest-neighbor framework for continual semi-supervised learning](https://mlsanity.com/paper/2212.05102/) (2022)
- [Semi-Supervised Learning with Multi-Head Co-Training](https://mlsanity.com/paper/2107.04795/) (2021)
- [Conservative-Progressive Collaborative Learning for Semi-supervised Semantic Segmentation](https://mlsanity.com/paper/2211.16701/) (2022)
- [Phaseformer: Phase-based Attention Mechanism for Underwater Image Restoration and Beyond](https://mlsanity.com/paper/2412.01456/) (2024)
- [SemiReward: A General Reward Model for Semi-supervised Learning](https://mlsanity.com/paper/2310.03013/) (2023)
- [FedCon: A Contrastive Framework for Federated Semi-Supervised Learning](https://mlsanity.com/paper/2109.04533/) (2021)
- [Latent Space Regularization for Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2104.02633/) (2021)
- [Uncertainty Estimation on Sequential Labeling via Uncertainty Transmission](https://mlsanity.com/paper/2311.08726/) (2023)
- [CDMAD: Class-Distribution-Mismatch-Aware Debiasing for Class-Imbalanced Semi-Supervised Learning](https://mlsanity.com/paper/2403.10391/) (2024)
- [LocalDrop: A Hybrid Regularization for Deep Neural Networks](https://mlsanity.com/paper/2103.00719/) (2021)
- [FedTriNet: A Pseudo Labeling Method with Three Players for Federated Semi-supervised Learning](https://mlsanity.com/paper/2109.05612/) (2021)
- [Robust Semi-supervised Federated Learning for Images Automatic Recognition in Internet of Drones](https://mlsanity.com/paper/2201.01230/) (2022)
- [Learning from Noisy Labels via Dynamic Loss Thresholding](https://mlsanity.com/paper/2104.02570/) (2021)
- [Not All Attention Is All You Need](https://mlsanity.com/paper/2104.04692/) (2021)
- [Road Scenes Segmentation Across Different Domains by Disentangling Latent Representations](https://mlsanity.com/paper/2108.03021/) (2021)
- [Boosting Semi-Supervised Learning with Contrastive Complementary Labeling](https://mlsanity.com/paper/2212.06643/) (2022)
- [Towards Understanding Why FixMatch Generalizes Better Than Supervised Learning](https://mlsanity.com/paper/2410.11206/) (2024)
- [Semi-Supervised Federated Learning with non-IID Data: Algorithm and System Design](https://mlsanity.com/paper/2110.13388/) (2021)
- [Semantic-Aware Mixup for Domain Generalization](https://mlsanity.com/paper/2304.05675/) (2023)

## Abstract (arXiv)

Recently, the training with adversarial examples, which are generated by adding a small but worst-case perturbation on input examples, has been proved to improve generalization performance of neural networks. In contrast to the individually biased inputs to enhance the generality, this paper introduces adversarial dropout, which is a minimal set of dropouts that maximize the divergence between the outputs from the network with the dropouts and the training supervisions. The identified adversarial dropout are used to reconfigure the neural network to train, and we demonstrated that training on the reconfigured sub-network improves the generalization performance of supervised and semi-supervised learning tasks on MNIST and CIFAR-10. We analyzed the trained model to reason the performance improvement, and we found that adversarial dropout increases the sparsity of neural networks more than the standard dropout does.