mlsanityData to 7 Oct 2026

Holistic Adversarially Robust Pruning

arXiv 2412.14714 ↗First posted 19 Dec 2024cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Neural networks can be drastically shrunk in size by removing redundant parameters. While crucial for the deployment on resource-constraint hardware, oftentimes, compression comes with a severe drop in accuracy and lack of adversarial robustness. Despite recent advances, counteracting both aspects has only succeeded for moderate compression rates so far. We propose a novel method, HARP, that copes with aggressive pruning significantly better than prior work. For this, we consider the network holistically. We learn a global compression strategy that optimizes how many parameters (compression rate) and which parameters (scoring connections) to prune specific to each layer individually. Our method fine-tunes an existing model with dynamic regularization, that follows a step-wise incremental function balancing the different objectives. It starts by favoring robustness before shifting focus on reaching the target compression rate and only then handles the objectives equally. The learned compression strategies allow us to maintain the pre-trained model natural accuracy and its adversarial robustness for a reduction by 99% of the network original size. Moreover, we observe a crucial influence of non-uniform compression across layers.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 34 of its 53 references that are in the map. First posted on arXiv 19 Dec 2024.

Builds on

34 of 53 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
22 more
  1. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  2. 2019Adversarial Training for Free!cited by 638
  3. 2018Rethinking the Value of Network Pruningcited by 595
  4. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  5. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  6. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  7. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  8. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
  9. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  10. 2020Bag of Tricks for Adversarial Trainingcited by 199
  11. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  12. 2016Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruningcited by 133
  13. 2018Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attackcited by 123
  14. 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 122
  15. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  16. 2020A Survey on Text Classification: From Shallow to Deep Learningcited by 80
  17. 2020Do Wider Neural Networks Really Help Adversarial Robustness?cited by 79
  18. 2018Provable Robustness of ReLU networks via Maximization of Linear Regionscited by 51
  19. 2020A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNscited by 37
  20. 2018Learning to Prune Filters in Convolutional Neural Networkscited by 25
  21. 2022Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Networkcited by 21
  22. 2022Understanding the effect of sparsity on neural networks robustnesscited by 8

Built on by

Cited by year

Citing papers per year: 2025: 5.

5 papers.

  1. 2025On the Interaction of Compressibility and Adversarial Robustnesscited by 2
  2. 2025Two is Better than One: Efficient Ensemble Defense for Robust and Compact Modelscited by 1
  3. 2025Adversarially Robust Spiking Neural Networks with Sparse Connectivitycited by 0
  4. 2025S2AP: Score-space Sharpness Minimization for Adversarial Pruningcited by 0
  5. 2025Lipschitz-aware Linearity Grafting for Certified Robustnesscited by 0