mlsanityData to 7 Oct 2026

Revisiting Random Channel Pruning for Neural Network Compression

arXiv 2205.05676 ↗First posted 11 May 2022cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

Channel (or 3D filter) pruning serves as an effective way to accelerate the inference of neural networks. There has been a flurry of algorithms that try to solve this practical problem, each being claimed effective in some ways. Yet, a benchmark to compare those algorithms directly is lacking, mainly due to the complexity of the algorithms and some custom settings such as the particular network configuration or training procedure. A fair benchmark is important for the further development of channel pruning. Meanwhile, recent investigations reveal that the channel configurations discovered by pruning algorithms are at least as important as the pre-trained weights. This gives channel pruning a new role, namely searching the optimal channel configuration. In this paper, we try to determine the channel configuration of the pruned models by random search. The proposed approach provides a new way to compare different methods, namely how well they behave compared with random pruning. We show that this simple strategy works quite well compared with other channel pruning methods. We also show that under this setting, there are surprisingly no clear winners among different channel importance evaluation methods, which then may tilt the research efforts into advanced channel configuration searching methods.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 53 of its 74 references that are in the map. First posted on arXiv 11 May 2022.

Builds on

53 of 74 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  8. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  9. 2018Rethinking the Value of Network Pruningcited by 595
  10. 2020What is the State of Neural Network Pruning?cited by 578
  11. 2019Importance Estimation for Neural Network Pruningcited by 531
  12. 2019Similarity-Preserving Knowledge Distillationcited by 507
38 more
  1. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  3. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  4. 2015Deep Learning with Limited Numerical Precisioncited by 368
  5. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  6. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  7. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  8. 2018Slimmable Neural Networkscited by 301
  9. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  10. 2014Speeding up Convolutional Neural Networks with Low Rank Expansionscited by 283
  11. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  12. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  13. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  14. 2019Universally Slimmable Networks and Improved Training Techniquescited by 219
  15. 2016Trained Ternary Quantizationcited by 203
  16. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  17. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  18. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 148
  19. 2019Towards Optimal Structured CNN Pruning via Generative Adversarial Learningcited by 145
  20. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  21. 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
  22. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  23. 2020ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgettingcited by 85
  24. 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
  25. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  26. 2020Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compressioncited by 72
  27. 2019Learning Filter Basis for Convolutional Neural Network Compressioncited by 57
  28. 2020DHP: Differentiable Meta Pruning via HyperNetworkscited by 57
  29. 2019Neural Network Pruning with Residual-Connections and Limited-Datacited by 52
  30. 2020Discrimination-aware Network Pruning for Deep Model Compressioncited by 50
  31. 2018Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compressioncited by 40
  32. 2019XNAS: Neural Architecture Search with Expert Advicecited by 35
  33. 2020Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruningcited by 29
  34. 2020MixMix: All You Need for Data-Free Compression Are Feature and Data Mixingcited by 29
  35. 2020Auto Graph Encoder-Decoder for Neural Network Pruningcited by 26
  36. 2019SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networkscited by 17
  37. 2020AOWS: Adaptive and optimal network width search with latency constraintscited by 17
  38. 2020The Heterogeneity Hypothesis: Finding Layer-Wise Differentiated Network Architecturescited by 13

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 24; 2024: 10; 2025: 10; 2026: 4.

50 papers.

Sort:

By area:

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
  3. 2023EVC: Towards Real-Time Neural Image Compression with Mask Decaycited by 46
  4. 2023OTOV2: Automatic, Generic, User-Friendlycited by 32
  5. 2023Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?cited by 20
  6. 2024Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratchcited by 20
  7. 2023Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspectivecited by 17
  8. 2024Lightweight Pixel Difference Networks for Efficient Visual Representation Learningcited by 17
  9. 2024PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptationcited by 17
  10. 2023Differentiable Transportation Pruningcited by 10
  11. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  12. 2024Self-Adapting Large Visual-Language Models to Edge Devices across Visual Modalitiescited by 7
38 more
  1. 2024Is Oracle Pruning the True Oracle?cited by 7
  2. 2023WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networkscited by 6
  3. 2023OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operatorscited by 6
  4. 2023Dynamic Structure Pruning for Compressing CNNscited by 5
  5. 2025Efficient LLMs with AMP: Attention Heads and MLP Pruningcited by 5
  6. 2022Can We Find Strong Lottery Tickets in Generative Models?cited by 4
  7. 2023UPSCALE: Unconstrained Channel Pruningcited by 4
  8. 2023Accelerating Learnt Video Codecs with Gradient Decay and Layer-wise Distillationcited by 4
  9. 2023Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networkscited by 3
  10. 2023SCoTTi: Save Computation at Training Time with an adaptive frameworkcited by 3
  11. 2025One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compressioncited by 3
  12. 2023Practical Network Acceleration with Tiny Sets: Hypothesis, Theory, and Algorithmcited by 2
  13. 2023Pruning at Initialization -- A Sketching Perspectivecited by 2
  14. 2023Edge-InversionNet: Enabling Efficient Inference of InversionNet on Edge Devicescited by 2
  15. 2024Shapley Pruning for Neural Network Compressioncited by 2
  16. 2025Towards Extreme Pruning of LLMs with Plug-and-Play Mixed Sparsitycited by 2
  17. 2023Canvas: End-to-End Kernel Architecture Search in Neural Networkscited by 1
  18. 2023The Sparsity Roofline: Understanding the Hardware Limits of Sparse Neural Networkscited by 1
  19. 2023Dynamic Shuffle: An Efficient Channel Mixture Methodcited by 1
  20. 2023Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environmentscited by 1
  21. 2024Characterizing Disparity Between Edge Models and High-Accuracy Base Models for Vision Taskscited by 1
  22. 2025One-Cycle Structured Pruning via Stability-Driven Subnetwork Searchcited by 1
  23. 2025Meta Pruning via Graph Metanetworks : A Universal Meta Learning Framework for Network Pruningcited by 1
  24. 2026MedCore: Boundary-Preserving Medical Core Pruning for MedSAMcited by 1
  25. 2022Optimizing Learning Rate Schedules for Iterative Pruning of Deep Neural Networkscited by 0
  26. 2023Rewarded meta-pruning: Meta Learning with Rewards for Channel Pruningcited by 0
  27. 2023Structured Pruning for Multi-Task Deep Neural Networkscited by 0
  28. 2023Knowledge Distillation Layer that Lets the Student Decidecited by 0
  29. 2024Towards Generalized Entropic Sparsification for Convolutional Neural Networkscited by 0
  30. 2024PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimatorcited by 0
  31. 2025Expanding-and-Shrinking Binary Neural Networkscited by 0
  32. 2025Pruning Everything, Everywhere, All at Oncecited by 0
  33. 2025FAIR-Pruner: A Flexible Framework for Automatic Layer-Wise Pruning via Tolerance of Differencecited by 0
  34. 2025TinyEcoWeedNet: Edge Efficient Real-Time Aerial Agricultural Weed Detectioncited by 0
  35. 2025C-SWAP: Explainability-Aware Structured Pruning for Efficient Neural Networks Compressioncited by 0
  36. 2026Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Informationcited by 0
  37. 2026S4oP: Operator-level Pruning of Structured State Space Models for Resource-Constrained Devicescited by 0
  38. 2026InterPruner: Interactive Structured Pruning via Taylor-Implicit Criterion and Language-Prior Modulator for Multimodal Object Detectioncited by 0