mlsanityData to 7 Oct 2026

Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions

arXiv 2209.08554 ↗First posted 18 Sep 2022cs.LG cs.AIcited by 20

Not on the map. Why?

Abstract (arXiv)

Pruning is one of the predominant approaches for compressing deep neural networks (DNNs). Lately, coresets (provable data summarizations) were leveraged for pruning DNNs, adding the advantage of theoretical guarantees on the trade-off between the compression rate and the approximation error. However, coresets in this domain were either data-dependent or generated under restrictive assumptions on both the model's weights and inputs. In real-world scenarios, such assumptions are rarely satisfied, limiting the applicability of coresets. To this end, we suggest a novel and robust framework for computing such coresets under mild assumptions on the model's weights and without any assumption on the training data. The idea is to compute the importance of each neuron in each layer with respect to the output of the following layer. This is achieved by a combination of L"owner ellipsoid and Caratheodory theorem. Our method is simultaneously data-independent, applicable to various networks and datasets (due to the simplified assumptions), and theoretically supported. Experimental results show that our method outperforms existing coreset based neural pruning approaches across a wide range of networks and datasets. For example, our method achieved a 62% compression rate on ResNet50 on ImageNet with 1.09% drop in accuracy.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 74 of its 127 references that are in the map. First posted on arXiv 18 Sep 2022.

Builds on

74 of 127 references.

References · 74

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  7. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  9. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  10. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  11. 2018Rethinking the Value of Network Pruningcited by 595
  12. 2020What is the State of Neural Network Pruning?cited by 578
38 more
  1. 2019Importance Estimation for Neural Network Pruningcited by 531
  2. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  3. 2019The State of Sparsity in Deep Neural Networkscited by 471
  4. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  5. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  6. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  7. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  8. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  9. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  10. 2018Slimmable Neural Networkscited by 301
  11. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  12. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  13. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  14. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  15. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  16. 2014Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decompositioncited by 199
  17. 2015Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applicationscited by 161
  18. 2015Compressing Neural Networks with the Hashing Trickcited by 150
  19. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  20. 2020Dynamic Model Pruning with Feedbackcited by 125
  21. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  22. 2015Convolutional neural networks with low-rank regularizationcited by 102
  23. 2018Solving Linear Programs in the Current Matrix Multiplication Timecited by 100
  24. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  25. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  26. 2018Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clusteringcited by 94
  27. 2015Fast ConvNets Using Group-wise Brain Damagecited by 81
  28. 2016Coresets for Scalable Bayesian Logistic Regressioncited by 79
  29. 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
  30. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  31. 2019Pruning from Scratchcited by 72
  32. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  33. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  34. 2018On Coresets for Logistic Regressioncited by 65
  35. 2016Coresets and Sketchescited by 63
  36. 2019Learning Filter Basis for Convolutional Neural Network Compressioncited by 57
  37. 2017Compression-aware Training of Deep Networkscited by 46
  38. 2018Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Boundscited by 43

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 5; 2025: 5; 2026: 3.

20 papers.

Sort:

By area:

  1. 2024Data-Efficient Learning via Clustering-Based Sensitivity Sampling: Foundation Models and Beyondcited by 15
  2. 2023Provable Data Subset Selection For Efficient Neural Network Trainingcited by 14
  3. 2023Dataset Distillation Meets Provable Subset Selectioncited by 7
  4. 2023Data-Efficient Training of CNNs and Transformers with Coresets: A Stability Perspectivecited by 6
  5. 2025DataS^3: Dataset Subset Selection for Specializationcited by 6
  6. 2023ORBSLAM3-Enhanced Autonomous Toy Drones: Pioneering Indoor Explorationcited by 5
  7. 2022Deep Learning on Home Drone: Searching for the Optimal Architecturecited by 4
  8. 2023AutoCoreset: An Automatic Practical Coreset Construction Frameworkcited by 4
  9. 2026Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Cachingcited by 2
  10. 2025Improving Model Classification by Optimizing the Training Datasetcited by 1
  11. 2025Compress to Impress: Efficient LLM Adaptation Using a Single Gradient Step on 100 Samplescited by 1
  12. 2025ModHiFi: Identifying High Fidelity predictive components for Model Modificationcited by 1
8 more
  1. 2026Provable Pruning for Efficient 3D Gaussian Splatting via Coresetscited by 1
  2. 2023Computing Approximate ℓp Sensitivitiescited by 0
  3. 2024PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimatorcited by 0
  4. 2024A Greedy Hierarchical Approach to Whole-Network Filter-Pruning in CNNscited by 0
  5. 2024Theoretically Grounded Pruning of Large Ground Sets for Constrained, Discrete Optimizationcited by 0
  6. 2024A Data-Free Analytical Quantization Scheme for Deep Learning Modelscited by 0
  7. 2025Pruning Everything, Everywhere, All at Oncecited by 0
  8. 2026Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exitcited by 0