mlsanityData to 7 Oct 2026

A Probabilistic Approach to Neural Network Pruning

arXiv 2105.10065 ↗First posted 20 May 2021cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Neural network pruning techniques reduce the number of parameters without compromising predicting ability of a network. Many algorithms have been developed for pruning both over-parameterized fully-connected networks (FCNs) and convolutional neural networks (CNNs), but analytical studies of capabilities and compression ratios of such pruned sub-networks are lacking. We theoretically study the performance of two pruning techniques (random and magnitude-based) on FCNs and CNNs. Given a target network {whose weights are independently sampled from appropriate distributions}, we provide a universal approach to bound the gap between a pruned and the target network in a probabilistic sense. The results establish that there exist pruned networks with expressive power within any specified bound from the target network.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 20 May 2021.

Builds on

30 of 51 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  11. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  12. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
18 more
  1. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
  2. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  3. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  4. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
  5. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  6. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  7. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  8. 2018The Singular Values of Convolutional Layerscited by 95
  9. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  10. 2019Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networkscited by 79
  11. 2020Logarithmic Pruning is All You Needcited by 71
  12. 2019Optimization for deep learning: theory and algorithmscited by 70
  13. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  14. 2018Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Boundscited by 43
  15. 2019A Selective Overview of Deep Learningcited by 31
  16. 2020Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruningcited by 23
  17. 2019SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networkscited by 17
  18. 2020Greedy Optimization Provably Wins the Lottery: Logarithmic Number of Winning Tickets is Enoughcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 0; 2025: 5; 2026: 1.

15 papers.

Sort:

  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  2. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  3. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  4. 2022Partial Model Averaging in Federated Learning: Performance Guarantees and Benefitscited by 8
  5. 2025Catch-Only-One: Non-Transferable Examples for Model-Specific Authorizationcited by 7
  6. 2025Re-Key-Free, Risky-Free: Adaptable Model Usage Controlcited by 6
  7. 2025FedRTS: Federated Robust Pruning via Combinatorial Thompson Samplingcited by 4
  8. 2023On Model Explanations with Transferable Neural Pathwayscited by 1
  9. 2025Theoretical Compression Bounds for Wide Multilayer Perceptronscited by 1
  10. 2026Elimination-compensation pruning for fully-connected neural networkscited by 1
  11. 2022Automating the Design and Development of Gradient Descent Trained Expert System Networkscited by 0
  12. 2023A Dataset Fusion Algorithm for Generalised Anomaly Detection in Homogeneous Periodic Time Series Datasetscited by 0
3 more
  1. 2023Generalization Bounds for Magnitude-Based Pruning via Sparse Matrix Sketchingcited by 0
  2. 2023To prune or not to prune : A chaos-causality approach to principled pruning of dense neural networkscited by 0
  3. 2025Explore and Establish Synergistic Effects Between Weight Pruning and Coreset Selection in Neural Network Trainingcited by 0