mlsanityData to 14 Oct 2026

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks

arXiv 2306.14306 ↗First posted 25 Jun 2023cs.LG cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Robustness and compactness are two essential attributes of deep learning models that are deployed in the real world. The goals of robustness and compactness may seem to be at odds, since robustness requires generalization across domains, while the process of compression exploits specificity in one domain. We introduce Adaptive Sharpness-Aware Pruning (AdaSAP), which unifies these goals through the lens of network sharpness. The AdaSAP method produces sparse networks that are robust to input variations which are unseen at training time. We achieve this by strategically incorporating weight perturbations in order to optimize the loss landscape. This allows the model to be both primed for pruning and regularized for improved robustness. AdaSAP improves the robust accuracy of pruned models on image classification by up to +6% on ImageNet C and +4% on ImageNet V2, and on object detection by +4% on a corrupted Pascal VOC dataset, over a wide range of compression ratios, pruning criteria, and network architectures, outperforming recent pruning art by large margins.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 64 of its 78 references that are in the map. First posted on arXiv 25 Jun 2023.

Builds on

64 of 78 references.

References · 64

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
38 more
  1. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  2. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  3. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  4. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  5. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  6. 2019Importance Estimation for Neural Network Pruningcited by 541
  7. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  8. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  10. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
  11. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  12. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  13. 2018How Does Batch Normalization Help Optimization?cited by 357
  14. 2018Slimmable Neural Networkscited by 305
  15. 2017Certified Defenses for Data Poisoning Attackscited by 303
  16. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  17. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  18. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 227
  19. 2018Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noisecited by 216
  20. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  21. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  22. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  23. 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 147
  24. 2022Understanding The Robustness in Vision Transformerscited by 144
  25. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  26. 2022Towards Efficient and Scalable Sharpness-Aware Minimizationcited by 129
  27. 2020Neural Pruning via Growing Regularizationcited by 97
  28. 2022Sharpness-Aware Training for Freecited by 95
  29. 2020Channel Pruning via Automatic Structure Searchcited by 85
  30. 2017Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapescited by 84
  31. 2020EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruningcited by 80
  32. 2022When Do Flat Minima Optimizers Work?cited by 78
  33. 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
  34. 2020Progressive Skeletonization: Trimming more fat from a network at initializationcited by 64
  35. 2021Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracycited by 62
  36. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  37. 2016Defensive Distillation is Not Robust to Adversarial Examplescited by 60
  38. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 5.

17 papers.

Sort:

By area:

  1. 2024Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustnesscited by 11
  2. 2024Effective Layer Pruning Through Similarity Metric Perspectivecited by 6
  3. 2024MoreauPruner: Robust Pruning of Large Language Models against Weight Perturbationscited by 5
  4. 2025SAFE: Finding Sparse and Flat Minima to Improve Pruningcited by 5
  5. 2026Cut Less, Fold More: Model Compression through the Lens of Projection Geometrycited by 4
  6. 2024More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routingcited by 2
  7. 2025FairSAM: Fair Classification on Corrupted Image Data Through Sharpness-Aware Minimizationcited by 1
  8. 2025Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruningcited by 1
  9. 2024Adaptive Pruning with Module Robustness Sensitivity: Balancing Compression and Robustnesscited by 0
  10. 2024Layer Pruning with Consensus: A Triple-Win Solutioncited by 0
  11. 2025Pruning Everything, Everywhere, All at Oncecited by 0
  12. 2025Pruning Cannot Hurt Robustness: Certified Trade-offs in Reinforcement Learningcited by 0
5 more
  1. 2025S2AP: Score-space Sharpness Minimization for Adversarial Pruningcited by 0
  2. 2026Towards Compact and Robust DNNs via Compression-aware Sharpness Minimizationcited by 0
  3. 2026Robustness Emerges Early in Training Dynamics, but Is Not Preservedcited by 0
  4. 2026Suppress and Diversify: Refining Robust Pathways for Corruption Robustnesscited by 0
  5. 2026Sparsity-Adaptive Sharpness-Aware Minimizationcited by 0