mlsanityData to 7 Oct 2026

SNIP: Single-shot Network Pruning based on Connection Sensitivity

arXiv 1810.02340 ↗First posted 4 Oct 2018cs.CV cs.LGcited by 802

Not on the map. Why?

Cited by 24 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pruning large neural networks while maintaining their performance is often desirable due to the reduced space and time complexity. In existing methods, pruning is done within an iterative optimization procedure with either heuristically designed pruning schedules or additional hyperparameters, undermining their utility. In this work, we present a new approach that prunes a given network once at initialization prior to training. To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task. This eliminates the need for both pretraining and the complex pruning schedule while making it robust to architecture variations. After pruning, the sparse network is trained in the standard way. Our method obtains extremely sparse networks with virtually the same accuracy as the reference network on the MNIST, CIFAR-10, and Tiny-ImageNet classification tasks and is broadly applicable to various architectures including convolutional, residual and recurrent networks. Unlike existing methods, our approach enables us to demonstrate that the retained connections are indeed relevant to the given task.

Key facts. Cited by 802 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 125; 2022: 137; 2023: 141; 2024: 170; 2025: 140; 2026: 89.

802 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  3. 2023LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPScited by 420
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  5. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 247
  6. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  7. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  8. 2021Accelerating Sparse Deep Neural Networkscited by 213
  9. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  10. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  11. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  12. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
38 more
  1. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  2. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  3. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  4. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 163
  5. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  6. 2021Learned Token Pruning for Transformerscited by 138
  7. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  8. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  9. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  10. 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 107
  11. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  12. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 88
  13. 2022The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Trainingcited by 85
  14. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  15. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 83
  16. 2022SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapterscited by 83
  17. 2022Unified Visual Transformer Compressioncited by 79
  18. 2022PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importancecited by 79
  19. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  20. 2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 76
  21. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  22. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  23. 2021Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Modelscited by 75
  24. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Frameworkcited by 70
  25. 2021Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracycited by 62
  26. 2023Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMscited by 61
  27. 2022The State of Sparse Training in Deep Reinforcement Learningcited by 60
  28. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  29. 2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 58
  30. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  31. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  32. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  33. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  34. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  35. 2023LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuningcited by 53
  36. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  37. 2021A Survey on Green Deep Learningcited by 52
  38. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 51

50 most cited of 802 All 802 (JSON)