mlsanityData to 7 Oct 2026

Pruning Adversarially Robust Neural Networks without Adversarial Examples

arXiv 2210.04311 ↗First posted 9 Oct 2022cs.LG cs.AI cs.CRcited by 12

Not on the map. Why?

Abstract (arXiv)

Adversarial pruning compresses models while preserving robustness. Current methods require access to adversarial examples during pruning. This significantly hampers training efficiency. Moreover, as new adversarial attacks and training methods develop at a rapid rate, adversarial pruning methods need to be modified accordingly to keep up. In this work, we propose a novel framework to prune a previously trained robust neural network while maintaining adversarial robustness, without further generating adversarial examples. We leverage concurrent self-distillation and pruning to preserve knowledge in the original model as well as regularizing the pruned model via the Hilbert-Schmidt Information Bottleneck. We comprehensively evaluate our proposed framework and show its superior performance in terms of both adversarial robustness and efficiency when pruning architectures trained on the MNIST, CIFAR-10, and CIFAR-100 datasets against five state-of-the-art attacks. Code is available at https://github.com/neu-spiral/PwoA/.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 9 Oct 2022.

Builds on

36 of 46 references.

References · 36

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2020Knowledge Distillation: A Surveycited by 1,578
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2016Deep Variational Information Bottleneckcited by 915
24 more
  1. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  2. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
  3. 2019Adversarial Training for Free!cited by 638
  4. 2018Rethinking the Value of Network Pruningcited by 595
  5. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  6. 2018Feature Denoising for Improving Adversarial Robustnesscited by 312
  7. 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
  8. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  9. 2021On the Convergence and Robustness of Adversarial Trainingcited by 127
  10. 2019Adversarially Robust Distillationcited by 124
  11. 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 118
  12. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  13. 2020Learnable Boundary Guided Adversarial Trainingcited by 93
  14. 2019The HSIC Bottleneck: Deep Learning without Back-Propagationcited by 77
  15. 2020The Conditional Entropy Bottleneckcited by 75
  16. 2018Sparse DNNs with Improved Adversarial Robustnesscited by 71
  17. 2019Model Compression with Adversarial Robustness: A Unified Optimization Frameworkcited by 71
  18. 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 71
  19. 2018Training for Faster Adversarial Robustness Verification via Inducing ReLU Stabilitycited by 67
  20. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  21. 2018ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multiplierscited by 42
  22. 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 41
  23. 2021Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustnesscited by 20
  24. 2019Are Self-Driving Cars Secure? Evasion Attacks against Deep Neural Networks for Steering Angle Predictioncited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 4; 2026: 1.

12 papers.

  1. 2023DualHSIC: HSIC-Bottleneck and Alignment for Continual Learningcited by 15
  2. 2024Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustnesscited by 10
  3. 2024End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Trainingcited by 8
  4. 2024Machine Learning Robustness: A Primercited by 8
  5. 2025Explaining Grokking and Information Bottleneck through Neural Collapse Emergencecited by 4
  6. 2024Non-transferable Pruningcited by 2
  7. 2024An Empirical Study of the Influence of Adversarial Fine-Tuning on Compressed Neural Networkscited by 1
  8. 2025Two is Better than One: Efficient Ensemble Defense for Robust and Compact Modelscited by 1
  9. 2024Adaptive Pruning with Module Robustness Sensitivity: Balancing Compression and Robustnesscited by 0
  10. 2025Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbationscited by 0
  11. 2025Hierarchical Sparse Plus Low Rank Compression of LLMcited by 0
  12. 2026GRAPE: Guided Parameter-Space Evolution for Compact Adversarial Robustnesscited by 0