# An Entropy-based Pruning Method for CNN Compression

arXiv: https://arxiv.org/abs/1706.05791 · HTML: https://mlsanity.com/paper/1706.05791/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2017.

## Built on by (most cited first, 30 of 37)

- [Structured Pruning for Deep Convolutional Neural Networks: A survey](https://mlsanity.com/paper/2303.00566/) (2023)
- [SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models](https://mlsanity.com/paper/2410.14720/) (2024)
- [Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?](https://mlsanity.com/paper/2308.06619/) (2023)
- [Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication](https://mlsanity.com/paper/2104.04268/) (2021)
- [Blending Pruning Criteria for Convolutional Neural Networks](https://mlsanity.com/paper/2107.05033/) (2021)
- [FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models](https://mlsanity.com/paper/2508.01506/) (2025)
- [Complexity-Driven CNN Compression for Resource-constrained Edge AI](https://mlsanity.com/paper/2208.12816/) (2022)
- [An Effective Information Theoretic Framework for Channel Pruning](https://mlsanity.com/paper/2408.16772/) (2024)
- [Developmental Plasticity-inspired Adaptive Pruning for Deep Spiking and Artificial Neural Networks](https://mlsanity.com/paper/2211.12714/) (2022)
- [Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning](https://mlsanity.com/paper/2209.13590/) (2022)
- [Efficient CNNs via Passive Filter Pruning](https://mlsanity.com/paper/2304.02319/) (2023)
- [Pruning for Feature-Preserving Circuits in CNNs](https://mlsanity.com/paper/2206.01627/) (2022)
- [Layer Collapse Can be Induced by Unstructured Pruning](https://mlsanity.com/paper/2404.16890/) (2024)
- [What is Left After Distillation? How Knowledge Transfer Impacts Fairness and Bias](https://mlsanity.com/paper/2410.08407/) (2024)
- [Pruning Before Training May Improve Generalization, Provably](https://mlsanity.com/paper/2301.00335/) (2023)
- [Stratified Knowledge-Density Super-Network for Scalable Vision Transformers](https://mlsanity.com/paper/2511.11683/) (2025)
- [Efficient Similarity-based Passive Filter Pruning for Compressing CNNs](https://mlsanity.com/paper/2210.17416/) (2022)
- [CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion](https://mlsanity.com/paper/2212.06335/) (2022)
- [Differentiable Weight Masks for Domain Transfer](https://mlsanity.com/paper/2308.13957/) (2023)
- [Mitigating severe over-parameterization in deep convolutional neural networks through forced feature abstraction and compression with an entropy-based heuristic](https://mlsanity.com/paper/2106.14190/) (2021)
- [Boosting the Convergence of Reinforcement Learning-based Auto-pruning Using Historical Data](https://mlsanity.com/paper/2107.08815/) (2021)
- [Edge AI for Internet of Energy: Challenges and Perspectives](https://mlsanity.com/paper/2311.16851/) (2023)
- [Decay Pruning Method: Smooth Pruning With a Self-Rectifying Procedure](https://mlsanity.com/paper/2406.03879/) (2024)
- [A Survey of Fragile Model Watermarking](https://mlsanity.com/paper/2406.04809/) (2024)
- [Routing the Lottery: Adaptive Subnetworks for Heterogeneous Data](https://mlsanity.com/paper/2601.22141/) (2026)
- [Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning](https://mlsanity.com/paper/2206.03596/) (2022)
- [Embedding Compression for Text Classification Using Dictionary Screening](https://mlsanity.com/paper/2211.12715/) (2022)
- [The Road to On-board Change Detection: A Lightweight Patch-Level Change Detection Network via Exploring the Potential of Pruning and Pooling](https://mlsanity.com/paper/2310.10166/) (2023)
- [Efficient Parameter Mining and Freezing for Continual Object Detection](https://mlsanity.com/paper/2402.12624/) (2024)
- [Towards Generalized Entropic Sparsification for Convolutional Neural Networks](https://mlsanity.com/paper/2404.04734/) (2024)

## Abstract (arXiv)

This paper aims to simultaneously accelerate and compress off-the-shelf CNN models via filter pruning strategy. The importance of each filter is evaluated by the proposed entropy-based method first. Then several unimportant filters are discarded to get a smaller CNN model. Finally, fine-tuning is adopted to recover its generalization ability which is damaged during filter pruning. Our method can reduce the size of intermediate activations, which would dominate most memory footprint during model training stage but is less concerned in previous compression methods. Experiments on the ILSVRC-12 benchmark demonstrate the effectiveness of our method. Compared with previous filter importance evaluation criteria, our entropy-based method obtains better performance. We achieve 3.3x speed-up and 16.64x compression on VGG-16, 1.54x acceleration and 1.47x compression on ResNet-50, both with about 1% top-5 accuracy decrease.