# Achieving Adversarial Robustness via Sparsity

arXiv: https://arxiv.org/abs/2009.05423 · HTML: https://mlsanity.com/paper/2009.05423/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustness](https://mlsanity.com/paper/2106.09129/) (2021)
- [Robust low-rank training via approximate orthonormal constraints](https://mlsanity.com/paper/2306.01485/) (2023)
- [Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity](https://mlsanity.com/paper/2310.18894/) (2023)
- [Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness](https://mlsanity.com/paper/2410.03030/) (2024)
- [Neural network relief: a pruning algorithm based on neural activity](https://mlsanity.com/paper/2109.10795/) (2021)
- [Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustness](https://mlsanity.com/paper/2409.01249/) (2024)
- [Relationship between Model Compression and Adversarial Robustness: A Review of Current Evidence](https://mlsanity.com/paper/2311.15782/) (2023)
- [An Adversarial Robustness Perspective on the Topology of Neural Networks](https://mlsanity.com/paper/2211.02675/) (2022)
- [iPINNs: Incremental learning for Physics-informed neural networks](https://mlsanity.com/paper/2304.04854/) (2023)
- [On the Interaction of Compressibility and Adversarial Robustness](https://mlsanity.com/paper/2507.17725/) (2025)
- [Finding Dynamics Preserving Adversarial Winning Tickets](https://mlsanity.com/paper/2202.06488/) (2022)
- [Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models](https://mlsanity.com/paper/2504.04747/) (2025)
- [Global Minimizers of ℓp-Regularized Objectives Yield the Sparsest ReLU Neural Networks](https://mlsanity.com/paper/2505.21791/) (2025)
- [Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models](https://mlsanity.com/paper/2603.05582/) (2026)

## Abstract (arXiv)

Network pruning has been known to produce compact models without much accuracy degradation. However, how the pruning process affects a network's robustness and the working mechanism behind remain unresolved. In this work, we theoretically prove that the sparsity of network weights is closely associated with model robustness. Through experiments on a variety of adversarial pruning methods, we find that weights sparsity will not hurt but improve robustness, where both weights inheritance from the lottery ticket and adversarial training improve model robustness in network pruning. Based on these findings, we propose a novel adversarial training method called inverse weights inheritance, which imposes sparse weights distribution on a large network by inheriting weights from a small network, thereby improving the robustness of the large network.