mlsanityData to 14 Oct 2026

Understanding the effect of sparsity on neural networks robustness

arXiv 2206.10915 ↗First posted 22 Jun 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper examines the impact of static sparsity on the robustness of a trained network to weight perturbations, data corruption, and adversarial examples. We show that, up to a certain sparsity achieved by increasing network width and depth while keeping the network capacity fixed, sparsified networks consistently match and often outperform their initially dense versions. Robustness and accuracy decline simultaneously for very high sparsity due to loose connectivity between network layers. Our findings show that a rapid robustness drop caused by network compression observed in the literature is due to a reduced network capacity rather than sparsity.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 22 Jun 2022.

Builds on

27 of 32 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  8. 2016Adversarial examples in the physical worldcited by 1,769
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  11. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  12. 2020What is the State of Neural Network Pruning?cited by 583
15 more
  1. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  2. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  3. 2019The State of Sparsity in Deep Neural Networkscited by 476
  4. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 414
  5. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 240
  6. 2019What Do Compressed Deep Neural Networks Forget?cited by 171
  7. 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 158
  8. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 138
  9. 2020Distributionally Robust Losses for Latent Covariate Mixturescited by 58
  10. 2020Towards Learning Convolutions from Scratchcited by 46
  11. 2020Neural networks with late-phase weightscited by 29
  12. 2020Are wider nets better given the same number of parameters?cited by 27
  13. 2020Entropic gradient descent algorithms and wide flat minimacited by 25
  14. 2021Quantization and Deployment of Deep Neural Networks on Microcontrollerscited by 21
  15. 2019Class-dependent Compression of Deep Neural Networkscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 2; 2025: 2.

8 papers.

  1. 2024Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustnesscited by 11
  2. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  3. 2022Studying the impact of magnitude pruning on contrastive learning methodscited by 7
  4. 2024Holistic Adversarially Robust Pruningcited by 5
  5. 2022Sparse Mixture Once-for-all Adversarial Training for Efficient In-Situ Trade-Off Between Accuracy and Robustness of DNNscited by 4
  6. 2023REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraintscited by 2
  7. 2025Two is Better than One: Efficient Ensemble Defense for Robust and Compact Modelscited by 1
  8. 2025Informed Mixing -- Improving Open Set Recognition via Attribution-based Augmentationcited by 0