# Defensive Dropout for Hardening Deep Neural Networks under Adversarial Attacks

arXiv: https://arxiv.org/abs/1809.05165 · HTML: https://mlsanity.com/paper/1809.05165/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2018.

## Built on by (most cited first, 13 of 13)

- [Deep Video Inpainting Detection](https://mlsanity.com/paper/2101.11080/) (2021)
- [Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention](https://mlsanity.com/paper/2311.17400/) (2023)
- [Improving the Transferability of Adversarial Examples with New Iteration Framework and Input Dropout](https://mlsanity.com/paper/2106.01617/) (2021)
- [Distilling Adversarial Robustness Using Heterogeneous Teachers](https://mlsanity.com/paper/2402.15586/) (2024)
- [The Fluorescent Veil: A Stealthy and Effective Physical Adversarial Patch Against Traffic Sign Recognition](https://mlsanity.com/paper/2409.12394/) (2024)
- [Sparta: Spatially Attentive and Adversarially Robust Activation](https://mlsanity.com/paper/2105.08269/) (2021)
- [Distribution-restrained Softmax Loss for the Model Robustness](https://mlsanity.com/paper/2303.12363/) (2023)
- [Towards Interpretable Ensemble Learning for Image-based Malware Detection](https://mlsanity.com/paper/2101.04889/) (2021)
- [Defensive Tensorization](https://mlsanity.com/paper/2110.13859/) (2021)
- [Learning Robust Kernel Ensembles with Kernel Average Pooling](https://mlsanity.com/paper/2210.00062/) (2022)
- [Stochastic-HMDs: Adversarial Resilient Hardware Malware Detectors through Voltage Over-scaling](https://mlsanity.com/paper/2103.06936/) (2021)
- [EMShepherd: Detecting Adversarial Samples via Side-channel Leakage](https://mlsanity.com/paper/2303.15571/) (2023)
- [AdROD: HyperNetwork-based Adversarially Robust Object Detection for Autonomous Driving](https://mlsanity.com/paper/2608.16031/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as any target labels. This work provides a solution to hardening DNNs under adversarial attacks through defensive dropout. Besides using dropout during training for the best test accuracy, we propose to use dropout also at test time to achieve strong defense effects. We consider the problem of building robust DNNs as an attacker-defender two-player game, where the attacker and the defender know each others' strategies and try to optimize their own strategies towards an equilibrium. Based on the observations of the effect of test dropout rate on test accuracy and attack success rate, we propose a defensive dropout algorithm to determine an optimal test dropout rate given the neural network model and the attacker's strategy for generating adversarial examples.We also investigate the mechanism behind the outstanding defense effects achieved by the proposed defensive dropout. Comparing with stochastic activation pruning (SAP), another defense method through introducing randomness into the DNN model, we find that our defensive dropout achieves much larger variances of the gradients, which is the key for the improved defense effects (much lower attack success rate). For example, our defensive dropout can reduce the attack success rate from 100% to 13.89% under the currently strongest attack i.e., C&W attack on MNIST dataset.