# Towards Practical Lottery Ticket Hypothesis for Adversarial Training

arXiv: https://arxiv.org/abs/2003.05733 · HTML: https://mlsanity.com/paper/2003.05733/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2020.

## Built on by (most cited first, 9 of 9)

- [Sparsity Winning Twice: Better Robust Generalization from More Efficient Training](https://mlsanity.com/paper/2202.09844/) (2022)
- [Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks](https://mlsanity.com/paper/2110.14068/) (2021)
- [Improving Robustness by Enhancing Weak Subnets](https://mlsanity.com/paper/2201.12765/) (2022)
- [Towards Lightweight Super-Resolution with Dual Regression Learning](https://mlsanity.com/paper/2207.07929/) (2022)
- [Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustness](https://mlsanity.com/paper/2409.01249/) (2024)
- [Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworks](https://mlsanity.com/paper/2310.15444/) (2023)
- [Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learning](https://mlsanity.com/paper/2304.11834/) (2023)
- [Finding Dynamics Preserving Adversarial Winning Tickets](https://mlsanity.com/paper/2202.06488/) (2022)
- [Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models](https://mlsanity.com/paper/2504.04747/) (2025)

## Abstract (arXiv)

Recent research has proposed the lottery ticket hypothesis, suggesting that for a deep neural network, there exist trainable sub-networks performing equally or better than the original model with commensurate training steps. While this discovery is insightful, finding proper sub-networks requires iterative training and pruning. The high cost incurred limits the applications of the lottery ticket hypothesis. We show there exists a subset of the aforementioned sub-networks that converge significantly faster during the training process and thus can mitigate the cost issue. We conduct extensive experiments to show such sub-networks consistently exist across various model structures for a restrictive setting of hyperparameters (e.g., carefully selected learning rate, pruning ratio, and model capacity). As a practical application of our findings, we demonstrate that such sub-networks can help in cutting down the total time of adversarial training, a standard approach to improve robustness, by up to 49% on CIFAR-10 to achieve the state-of-the-art robustness.