# Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression

arXiv: https://arxiv.org/abs/1812.04368 · HTML: https://mlsanity.com/paper/1812.04368/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2018.

## Built on by (most cited first, 30 of 40)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Only Train Once: A One-Shot Neural Network Training And Pruning Framework](https://mlsanity.com/paper/2107.07467/) (2021)
- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [OTOV2: Automatic, Generic, User-Friendly](https://mlsanity.com/paper/2303.06862/) (2023)
- [Multi-Scale Aligned Distillation for Low-Resolution Detection](https://mlsanity.com/paper/2109.06875/) (2021)
- [CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters](https://mlsanity.com/paper/2203.15331/) (2022)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression](https://mlsanity.com/paper/2502.16638/) (2025)
- [Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation](https://mlsanity.com/paper/2103.09716/) (2021)
- [Survey: Exploiting Data Redundancy for Optimization of Deep Learning](https://mlsanity.com/paper/2208.13363/) (2022)
- [OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operators](https://mlsanity.com/paper/2312.09411/) (2023)
- [Probabilistic Selective Encryption of Convolutional Neural Networks for Hierarchical Services](https://mlsanity.com/paper/2105.12344/) (2021)
- [Sauron U-Net: Simple automated redundancy elimination in medical image segmentation via filter pruning](https://mlsanity.com/paper/2209.13590/) (2022)
- [Dynamic Structure Pruning for Compressing CNNs](https://mlsanity.com/paper/2303.09736/) (2023)
- [CFDP: Common Frequency Domain Pruning](https://mlsanity.com/paper/2306.04147/) (2023)
- [Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation](https://mlsanity.com/paper/2406.12516/) (2024)
- [Pruning for Feature-Preserving Circuits in CNNs](https://mlsanity.com/paper/2206.01627/) (2022)
- [REPrune: Channel Pruning via Kernel Representative Selection](https://mlsanity.com/paper/2402.17862/) (2024)
- [Exploring Linear Feature Disentanglement For Neural Networks](https://mlsanity.com/paper/2203.11700/) (2022)
- [Edge-featured Graph Neural Architecture Search](https://mlsanity.com/paper/2109.01356/) (2021)
- [Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning](https://mlsanity.com/paper/2109.14729/) (2021)
- [CAT: Learning to Collaborate Channel and Spatial Attention from Multi-Information Fusion](https://mlsanity.com/paper/2212.06335/) (2022)
- [Filter Pruning based on Information Capacity and Independence](https://mlsanity.com/paper/2303.03645/) (2023)
- [Shapley Pruning for Neural Network Compression](https://mlsanity.com/paper/2407.15875/) (2024)
- [Maximum Redundancy Pruning: A Principle-Driven Layerwise Sparsity Allocation for LLMs](https://mlsanity.com/paper/2503.18377/) (2025)
- [Open-World Distributed Robot Self-Localization with Transferable Visual Vocabulary and Both Absolute and Relative Features](https://mlsanity.com/paper/2109.04569/) (2021)
- [Evolving Transferable Neural Pruning Functions](https://mlsanity.com/paper/2110.10876/) (2021)
- [Towards Efficient Target-Level Machine Unlearning Based on Essential Graph](https://mlsanity.com/paper/2406.10954/) (2024)
- [Differentiable Neural Architecture Learning for Efficient Neural Network Design](https://mlsanity.com/paper/2103.02126/) (2021)

## Abstract (arXiv)

Compressing convolutional neural networks (CNNs) has received ever-increasing research focus. However, most existing CNN compression methods do not interpret their inherent structures to distinguish the implicit redundancy. In this paper, we investigate the problem of CNN compression from a novel interpretable perspective. The relationship between the input feature maps and 2D kernels is revealed in a theoretical framework, based on which a kernel sparsity and entropy (KSE) indicator is proposed to quantitate the feature map importance in a feature-agnostic manner to guide model compression. Kernel clustering is further conducted based on the KSE indicator to accomplish high-precision CNN compression. KSE is capable of simultaneously compressing each layer in an efficient way, which is significantly faster compared to previous data-driven feature map pruning methods. We comprehensively evaluate the compression and speedup of the proposed method on CIFAR-10, SVHN and ImageNet 2012. Our method demonstrates superior performance gains over previous ones. In particular, it achieves 4.7 × FLOPs reduction and 2.9 × compression on ResNet-50 with only a Top-5 accuracy drop of 0.35% on ImageNet 2012, which significantly outperforms state-of-the-art methods.