# Adversarial Robustness vs Model Compression, or Both?

arXiv: https://arxiv.org/abs/1903.12561 · HTML: https://mlsanity.com/paper/1903.12561/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2019.

## Built on by (most cited first, 30 of 98)

- [GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://mlsanity.com/paper/2104.08826/) (2021)
- [Towards Robust Vision Transformer](https://mlsanity.com/paper/2105.07926/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [MedViT: A Robust Vision Transformer for Generalized Medical Image Classification](https://mlsanity.com/paper/2302.09462/) (2023)
- [The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training](https://mlsanity.com/paper/2202.02643/) (2022)
- [Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy](https://mlsanity.com/paper/2103.03014/) (2021)
- [Sparsity Winning Twice: Better Robust Generalization from More Efficient Training](https://mlsanity.com/paper/2202.09844/) (2022)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Visual Prompting for Adversarial Robustness](https://mlsanity.com/paper/2210.06284/) (2022)
- [Robust Predictable Control](https://mlsanity.com/paper/2109.03214/) (2021)
- [Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks](https://mlsanity.com/paper/2110.14068/) (2021)
- [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers](https://mlsanity.com/paper/2302.02596/) (2023)
- [LAFEAT: Piercing Through Adversarial Defenses with Latent Features](https://mlsanity.com/paper/2104.09284/) (2021)
- [On the Effect of Pruning on Adversarial Robustness](https://mlsanity.com/paper/2108.04890/) (2021)
- [BiBench: Benchmarking and Analyzing Network Binarization](https://mlsanity.com/paper/2301.11233/) (2023)
- [Pruning for Robust Concept Erasing in Diffusion Models](https://mlsanity.com/paper/2405.16534/) (2024)
- [Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network](https://mlsanity.com/paper/2204.02738/) (2022)
- [Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness](https://mlsanity.com/paper/2106.02734/) (2021)
- [ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Models](https://mlsanity.com/paper/2406.07831/) (2024)
- [Stealthy Backdoors as Compression Artifacts](https://mlsanity.com/paper/2104.15129/) (2021)
- [Robust Lottery Tickets for Pre-trained Language Models](https://mlsanity.com/paper/2211.03013/) (2022)
- [Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free](https://mlsanity.com/paper/2205.11819/) (2022)
- [Can pruning improve certified robustness of neural networks?](https://mlsanity.com/paper/2206.07311/) (2022)
- [Robust low-rank training via approximate orthonormal constraints](https://mlsanity.com/paper/2306.01485/) (2023)
- [Adversarial Examples Are Not Bugs, They Are Superposition](https://mlsanity.com/paper/2508.17456/) (2025)
- [MIA-Former: Efficient and Robust Vision Transformers via Multi-grained Input-Adaptation](https://mlsanity.com/paper/2112.11542/) (2021)
- [Pruning Adversarially Robust Neural Networks without Adversarial Examples](https://mlsanity.com/paper/2210.04311/) (2022)
- [AdaSTE: An Adaptive Straight-Through Estimator to Train Binary Neural Networks](https://mlsanity.com/paper/2112.02880/) (2021)

## Abstract (arXiv)

It is well known that deep neural networks (DNNs) are vulnerable to adversarial attacks, which are implemented by adding crafted perturbations onto benign examples. Min-max robust optimization based adversarial training can provide a notion of security against adversarial attacks. However, adversarial robustness requires a significantly larger capacity of the network than that for the natural training with only benign examples. This paper proposes a framework of concurrent adversarial training and weight pruning that enables model compression while still preserving the adversarial robustness and essentially tackles the dilemma of adversarial training. Furthermore, this work studies two hypotheses about weight pruning in the conventional setting and finds that weight pruning is essential for reducing the network model size in the adversarial setting, training a small model from scratch even with inherited initialization from the large model cannot achieve both adversarial robustness and high standard accuracy. Code is available at https://github.com/yeshaokai/Robustness-Aware-Pruning-ADMM.