mlsanityData to 14 Oct 2026

A Comprehensive Study of Structural Pruning for Vision Models

arXiv 2406.12315 ↗First posted 18 Jun 2024cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Structural pruning has emerged as a promising approach for producing more efficient models. Nevertheless, the community suffers from a lack of standardized benchmarks and metrics, leaving the progress in this area not fully comprehended. To fill this gap, we present the first comprehensive benchmark, termed PruningBench, for structural pruning. PruningBench showcases the following three characteristics: 1) PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques; 2) PruningBench systematically evaluates 16 existing pruning methods, encompassing a wide array of models (e.g., CNNs and ViTs) and tasks (e.g., classification and detection); 3) PruningBench provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards. We provide an online pruning platform for customizing pruning tasks and reproducing all results in this paper. Leaderboard results can also be available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 58 of its 79 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

58 of 79 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  8. 2020Knowledge Distillation: A Surveycited by 1,603
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  10. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  12. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
38 more
  1. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
  2. 2020What is the State of Neural Network Pruning?cited by 583
  3. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
  4. 2019Importance Estimation for Neural Network Pruningcited by 541
  5. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
  6. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
  7. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 315
  8. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 306
  9. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  10. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  11. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  12. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
  13. 2023DepGraph: Towards Any Structural Pruningcited by 203
  14. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 183
  15. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  16. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  17. 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 126
  18. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  19. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  20. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  21. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  22. 2020Neural Pruning via Growing Regularizationcited by 97
  23. 2020ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgettingcited by 85
  24. 2020EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruningcited by 80
  25. 2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 78
  26. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  27. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  28. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  29. 2022CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Predictioncited by 39
  30. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  31. 2019Approximated Oracle Filter Pruning for Destructive CNN Width Optimizationcited by 35
  32. 2022Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradientscited by 34
  33. 2022DropNet: Reducing Neural Network Complexity via Iterative Pruningcited by 29
  34. 2018Coreset-Based Neural Network Compressioncited by 27
  35. 2020Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Frameworkcited by 27
  36. 2022Winning the Lottery Ahead of Time: Efficient Early Network Pruningcited by 21
  37. 2020EDropout: Energy-Based Dropout and Pruning of Deep Neural Networkscited by 20
  38. 2022CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Modelscited by 20

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.