# MINT: Deep Network Compression via Mutual Information-based Neuron Trimming

arXiv: https://arxiv.org/abs/2003.08472 · HTML: https://mlsanity.com/paper/2003.08472/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2020.

## Built on by (most cited first, 11 of 11)

- [HRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels](https://mlsanity.com/paper/2202.10716/) (2022)
- [S-Rocket: Selective Random Convolution Kernels for Time Series Classification](https://mlsanity.com/paper/2203.03445/) (2022)
- [DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation](https://mlsanity.com/paper/2406.08804/) (2024)
- [Filter Pruning based on Information Capacity and Independence](https://mlsanity.com/paper/2303.03645/) (2023)
- [MI-PRUN: Optimize Large Language Model Pruning via Mutual Information](https://mlsanity.com/paper/2601.07212/) (2026)
- [Towards Explaining Deep Neural Network Compression Through a Probabilistic Latent Space](https://mlsanity.com/paper/2403.00155/) (2024)
- [MPruner: Optimizing Neural Network Size with CKA-Based Mutual Information Pruning](https://mlsanity.com/paper/2408.13482/) (2024)
- [Information Consistent Pruning: How to Efficiently Search for Sparse Networks?](https://mlsanity.com/paper/2501.15592/) (2025)
- [On Causal Inference for Data-free Structured Pruning](https://mlsanity.com/paper/2112.10229/) (2021)
- [On The Energy Statistics of Feature Maps in Pruning of Neural Networks with Skip-Connections](https://mlsanity.com/paper/2201.11209/) (2022)
- [Compressive Feature Selection for Remote Visual Multi-Task Inference](https://mlsanity.com/paper/2405.09077/) (2024)

## Abstract (arXiv)

Most approaches to deep neural network compression via pruning either evaluate a filter's importance using its weights or optimize an alternative objective function with sparsity constraints. While these methods offer a useful way to approximate contributions from similar filters, they often either ignore the dependency between layers or solve a more difficult optimization objective than standard cross-entropy. Our method, Mutual Information-based Neuron Trimming (MINT), approaches deep compression via pruning by enforcing sparsity based on the strength of the relationship between filters of adjacent layers, across every pair of layers. The relationship is calculated using conditional geometric mutual information which evaluates the amount of similar information exchanged between the filters using a graph-based criterion. When pruning a network, we ensure that retained filters contribute the majority of the information towards succeeding layers which ensures high performance. Our novel approach outperforms existing state-of-the-art compression-via-pruning methods on the standard benchmarks for this task: MNIST, CIFAR-10, and ILSVRC2012, across a variety of network architectures. In addition, we discuss our observations of a common denominator between our pruning methodology's response to adversarial attacks and calibration statistics when compared to the original network.