mlsanityData to 7 Oct 2026

Group Fisher Pruning for Practical Network Compression

arXiv 2108.00708 ↗First posted 2 Aug 2021cs.CV cs.LGcited by 130

Not on the map. Why?

Abstract (arXiv)

Network compression has been widely studied since it is able to reduce the memory and computation cost during inference. However, previous methods seldom deal with complicated structures like residual connections, group/depth-wise convolution and feature pyramid network, where channels of multiple layers are coupled and need to be pruned simultaneously. In this paper, we present a general channel pruning approach that can be applied to various complicated structures. Particularly, we propose a layer grouping algorithm to find coupled channels automatically. Then we derive a unified metric based on Fisher information to evaluate the importance of a single channel and coupled channels. Moreover, we find that inference speedup on GPUs is more correlated with the reduction of memory rather than FLOPs, and thus we employ the memory reduction of each channel to normalize the importance. Our method can be used to prune any structures including those with coupled channels. We conduct extensive experiments on various backbones, including the classic ResNet and ResNeXt, mobile-friendly MobileNetV2, and the NAS-based RegNet, both on image classification and object detection which is under-explored. Experimental results validate that our method can effectively prune sophisticated networks, boosting inference speed without sacrificing accuracy.

Key facts. Cited by 130 papers in the map (data through 7 Oct 2026). Builds on 42 of its 53 references that are in the map. First posted on arXiv 2 Aug 2021.

Builds on

42 of 53 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2020Designing Network Design Spacescited by 954
30 more
  1. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  2. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  3. 2019Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selectioncited by 539
  4. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  5. 2019Importance Estimation for Neural Network Pruningcited by 531
  6. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  7. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  8. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  9. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  10. 2018Slimmable Neural Networkscited by 301
  11. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  12. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  13. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  14. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  15. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  16. 2021Accelerating Sparse Deep Neural Networkscited by 213
  17. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  18. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  19. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  20. 2020Probabilistic Anchor Assignment with IoU Prediction for Object Detectioncited by 150
  21. 2019Feature Selective Anchor-Free Module for Single-Shot Object Detectioncited by 139
  22. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  23. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  24. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  25. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 92
  26. 2015Fast ConvNets Using Group-wise Brain Damagecited by 81
  27. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  28. 2019Neural Network Pruning with Residual-Connections and Limited-Datacited by 52
  29. 2019Approximated Oracle Filter Pruning for Destructive CNN Width Optimizationcited by 35
  30. 2019Benanza: Automatic μBenchmark Generation to Compute "Lower-bound" Latency and Inform Optimizations of Deep Learning Models on GPUscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 14; 2023: 25; 2024: 40; 2025: 25; 2026: 21.

130 papers.

Sort:

By area:

  1. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  2. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  3. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  4. 2021Training Neural Networks with Fixed Sparse Maskscited by 212
  5. 2023DepGraph: Towards Any Structural Pruningcited by 202
  6. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  7. 2023Structural Pruning for Diffusion Modelscited by 193
  8. 2022Language model compression with weighted low-rank factorizationcited by 192
  9. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  10. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  11. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  12. 2023Parameter-Efficient Fine-Tuning without Introducing New Latencycited by 59
38 more
  1. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 59
  2. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 57
  3. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
  4. 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
  5. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  6. 2023Adaptive Sparse Convolutional Networks with Global Context Enhancement for Faster Object Detection on Drone Imagescited by 41
  7. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  8. 2023Dynamic Sparse Training with Structured Sparsitycited by 36
  9. 2022SPDY: Accurate Pruning with Speedup Guaranteescited by 34
  10. 2021SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruningcited by 30
  11. 2024SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Modelscited by 24
  12. 2023Zero-Shot Neural Architecture Search: Challenges, Solutions, and Opportunitiescited by 21
  13. 2024A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Expertscited by 21
  14. 2024Isomorphic Pruning for Vision Modelscited by 21
  15. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
  16. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Modelscited by 19
  17. 2023Dynamic Sparsity Is Channel-Level Sparsity Learnercited by 17
  18. 2024OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimizationcited by 17
  19. 2022Numerical Optimizations for Weighted Low-rank Estimation on Language Modelcited by 14
  20. 2023Towards Green AI in Fine-tuning Large Language Models via Adaptive Backpropagationcited by 12
  21. 2025AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reusecited by 11
  22. 2023CP3: Channel Pruning Plug-in for Point-based Networkscited by 10
  23. 2024Multilingual Brain Surgeon: Large Language Models Can be Compressed Leaving No Language Behindcited by 10
  24. 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
  25. 2023F3-Pruning: A Training-Free and Generalized Pruning Strategy towards Faster and Finer Text-to-Video Synthesiscited by 9
  26. 2022Compressing Models with Few Samples: Mimicking then Replacingcited by 8
  27. 2024Structurally Prune Anything: Any Architecture, Any Framework, Any Timecited by 8
  28. 2025CommonKV: Compressing KV Cache with Cross-layer Parameter Sharingcited by 8
  29. 2024An Effective Information Theoretic Framework for Channel Pruningcited by 7
  30. 2023Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbationscited by 6
  31. 2024Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradientcited by 6
  32. 2025Olica: Efficient Structured Pruning of Large Language Models without Retrainingcited by 6
  33. 2024Maxwell's Demon at Work: Efficient Pruning by Leveraging Saturation of Neuronscited by 5
  34. 2024MINI-LLM: Memory-Efficient Structured Pruning for Large Language Modelscited by 5
  35. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  36. 2025Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compressioncited by 5
  37. 2022Boosting Pruned Networks with Linear Over-parameterizationcited by 4
  38. 2022FedILC: Weighted Geometric Mean and Invariant Gradient Covariance for Federated Learning on Non-IID Datacited by 4

50 most cited of 130 All 130 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025GoPrune: Accelerated Structured Pruning with ℓ2,p-Norm Optimization
  2. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Framework
  3. 2019Structural Pruning in Deep Neural Networks: A Small-World Approach
  4. 2026Attribution-Guided and Coverage-Maximized Pruning for Structural MoE Compression
  5. 2023SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity
  6. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers
  7. 2026HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks
  8. 2025LinDeps: A Fine-tuning Free Post-Pruning Method to Remove Layer-Wise Linear Dependencies with Guaranteed Performance Preservation