# Convolution-Weight-Distribution Assumption: Rethinking the Criteria of Channel Pruning

arXiv: https://arxiv.org/abs/2004.11627 · HTML: https://mlsanity.com/paper/2004.11627/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 30 of 30)

- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [A Bayesian Federated Learning Framework with Online Laplace Approximation](https://mlsanity.com/paper/2102.01936/) (2021)
- [ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection](https://mlsanity.com/paper/2310.13545/) (2023)
- [On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solver](https://mlsanity.com/paper/2208.03680/) (2022)
- [Understanding Self-attention Mechanism via Dynamical System Perspective](https://mlsanity.com/paper/2308.09939/) (2023)
- [MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models](https://mlsanity.com/paper/2505.19959/) (2025)
- [The Lottery Ticket Hypothesis for Self-attention in Convolutional Neural Network](https://mlsanity.com/paper/2207.07858/) (2022)
- [Is Oracle Pruning the True Oracle?](https://mlsanity.com/paper/2412.00143/) (2024)
- [Weight Expansion: A New Perspective on Dropout and Generalization](https://mlsanity.com/paper/2201.09209/) (2022)
- [Mirror Gradient: Towards Robust Multimodal Recommender Systems via Exploring Flat Local Minima](https://mlsanity.com/paper/2402.11262/) (2024)
- [Effective Layer Pruning Through Similarity Metric Perspective](https://mlsanity.com/paper/2405.17081/) (2024)
- [MUDGUARD: Taming Malicious Majorities in Federated Learning using Privacy-Preserving Byzantine-Robust Clustering](https://mlsanity.com/paper/2208.10161/) (2022)
- [Switchable Self-attention Module](https://mlsanity.com/paper/2209.05680/) (2022)
- [Entropy Induced Pruning Framework for Convolutional Neural Networks](https://mlsanity.com/paper/2208.06660/) (2022)
- [Mix-Pooling Strategy for Attention Mechanism](https://mlsanity.com/paper/2208.10322/) (2022)
- [Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks](https://mlsanity.com/paper/2308.04672/) (2023)
- [FedFQ: Federated Learning with Fine-Grained Quantization](https://mlsanity.com/paper/2408.08977/) (2024)
- [BGDB: Bernoulli-Gaussian Decision Block with Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2409.13116/) (2024)
- [ASR: Attention-alike Structural Re-parameterization](https://mlsanity.com/paper/2304.06345/) (2023)
- [Sensitivity-Guided Framework for Pruned and Quantized Reservoir Computing Accelerators](https://mlsanity.com/paper/2603.08737/) (2026)
- [Attention Round for Post-Training Quantization](https://mlsanity.com/paper/2207.03088/) (2022)
- [Efficient Climate Simulation via Machine Learning Method](https://mlsanity.com/paper/2209.08151/) (2022)
- [Towards Deep Network Steganography: From Networks to Networks](https://mlsanity.com/paper/2307.03444/) (2023)
- [Generative adversarial learning with optimal input dimension and its adaptive generator architecture](https://mlsanity.com/paper/2405.03723/) (2024)
- [DeepVigor+: Scalable and Accurate Semi-Analytical Fault Resilience Analysis for Deep Neural Network](https://mlsanity.com/paper/2410.15742/) (2024)
- [CWP: Instance complexity weighted channel-wise soft masks for network pruning](https://mlsanity.com/paper/2209.03534/) (2022)
- [SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity](https://mlsanity.com/paper/2310.19509/) (2023)
- [Layer Pruning with Consensus: A Triple-Win Solution](https://mlsanity.com/paper/2411.14345/) (2024)
- [Exploring and Reshaping the Weight Distribution in LLM](https://mlsanity.com/paper/2509.00046/) (2025)
- [RecycleLoRA: Rank-Revealing QR-Based Dual-LoRA Subspace Adaptation for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2603.28142/) (2026)

## Abstract (arXiv)

Channel pruning is a popular technique for compressing convolutional neural networks (CNNs), where various pruning criteria have been proposed to remove the redundant filters. From our comprehensive experiments, we found two blind spots in the study of pruning criteria: (1) Similarity: There are some strong similarities among several primary pruning criteria that are widely cited and compared. According to these criteria, the ranks of filters'Importance Score are almost identical, resulting in similar pruned structures. (2) Applicability: The filters'Importance Score measured by some pruning criteria are too close to distinguish the network redundancy well. In this paper, we analyze these two blind spots on different types of pruning criteria with layer-wise pruning or global pruning. The analyses are based on the empirical experiments and our assumption (Convolutional Weight Distribution Assumption) that the well-trained convolutional filters each layer approximately follow a Gaussian-alike distribution. This assumption has been verified through systematic and extensive statistical tests.