# Pruning Adversarially Robust Neural Networks without Adversarial Examples

arXiv: https://arxiv.org/abs/2210.04311 · HTML: https://mlsanity.com/paper/2210.04311/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 9 Oct 2022.

## Builds on (most cited first, 30 of 36)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Adversarial Training for Free!](https://mlsanity.com/paper/1904.12843/) (2019)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Feature Denoising for Improving Adversarial Robustness](https://mlsanity.com/paper/1812.03411/) (2018)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://mlsanity.com/paper/1705.07565/) (2017)
- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [Adversarially Robust Distillation](https://mlsanity.com/paper/1905.09747/) (2019)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [Adversarial Robustness vs Model Compression, or Both?](https://mlsanity.com/paper/1903.12561/) (2019)
- [Learnable Boundary Guided Adversarial Training](https://mlsanity.com/paper/2011.11164/) (2020)
- [The HSIC Bottleneck: Deep Learning without Back-Propagation](https://mlsanity.com/paper/1908.01580/) (2019)
- [The Conditional Entropy Bottleneck](https://mlsanity.com/paper/2002.05379/) (2020)
- [Sparse DNNs with Improved Adversarial Robustness](https://mlsanity.com/paper/1810.09619/) (2018)
- [Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey](https://mlsanity.com/paper/2007.00753/) (2020)
- [Model Compression with Adversarial Robustness: A Unified Optimization Framework](https://mlsanity.com/paper/1902.03538/) (2019)

## Built on by (most cited first, 12 of 12)

- [DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning](https://mlsanity.com/paper/2305.00380/) (2023)
- [Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustness](https://mlsanity.com/paper/2409.01249/) (2024)
- [End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training](https://mlsanity.com/paper/2402.09050/) (2024)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [Explaining Grokking and Information Bottleneck through Neural Collapse Emergence](https://mlsanity.com/paper/2509.20829/) (2025)
- [Non-transferable Pruning](https://mlsanity.com/paper/2410.08015/) (2024)
- [An Empirical Study of the Influence of Adversarial Fine-Tuning on Compressed Neural Networks](https://mlsanity.com/paper/2403.09441/) (2024)
- [Two is Better than One: Efficient Ensemble Defense for Robust and Compact Models](https://mlsanity.com/paper/2504.04747/) (2025)
- [Adaptive Pruning with Module Robustness Sensitivity: Balancing Compression and Robustness](https://mlsanity.com/paper/2410.15176/) (2024)
- [Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbations](https://mlsanity.com/paper/2506.04951/) (2025)
- [Hierarchical Sparse Plus Low Rank Compression of LLM](https://mlsanity.com/paper/2601.07839/) (2025)
- [GRAPE: Guided Parameter-Space Evolution for Compact Adversarial Robustness](https://mlsanity.com/paper/2606.14865/) (2026)

## Abstract (arXiv)

Adversarial pruning compresses models while preserving robustness. Current methods require access to adversarial examples during pruning. This significantly hampers training efficiency. Moreover, as new adversarial attacks and training methods develop at a rapid rate, adversarial pruning methods need to be modified accordingly to keep up. In this work, we propose a novel framework to prune a previously trained robust neural network while maintaining adversarial robustness, without further generating adversarial examples. We leverage concurrent self-distillation and pruning to preserve knowledge in the original model as well as regularizing the pruned model via the Hilbert-Schmidt Information Bottleneck. We comprehensively evaluate our proposed framework and show its superior performance in terms of both adversarial robustness and efficiency when pruning architectures trained on the MNIST, CIFAR-10, and CIFAR-100 datasets against five state-of-the-art attacks. Code is available at https://github.com/neu-spiral/PwoA/.