# Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning

arXiv: https://arxiv.org/abs/1912.08881 · HTML: https://mlsanity.com/paper/1912.08881/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2019.

## Built on by (most cited first, 30 of 89)

- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions](https://mlsanity.com/paper/2310.19775/) (2023)
- [Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy](https://mlsanity.com/paper/2106.13200/) (2021)
- [Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomy](https://mlsanity.com/paper/2305.06360/) (2023)
- [Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement](https://mlsanity.com/paper/2203.08008/) (2022)
- [Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces](https://mlsanity.com/paper/2212.14855/) (2022)
- [Discovering Transferable Forensic Features for CNN-generated Images Detection](https://mlsanity.com/paper/2208.11342/) (2022)
- [LION: Implicit Vision Prompt Tuning](https://mlsanity.com/paper/2303.09992/) (2023)
- [Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers](https://mlsanity.com/paper/2402.04744/) (2024)
- [ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNs](https://mlsanity.com/paper/2109.04236/) (2021)
- [ECLAD: Extracting Concepts with Local Aggregated Descriptors](https://mlsanity.com/paper/2206.04531/) (2022)
- [Effective dimension of machine learning models](https://mlsanity.com/paper/2112.04807/) (2021)
- [PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs](https://mlsanity.com/paper/2312.15230/) (2023)
- [Structured Pruning Adapters](https://mlsanity.com/paper/2211.10155/) (2022)
- [Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps](https://mlsanity.com/paper/2209.02869/) (2022)
- [Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask](https://mlsanity.com/paper/2209.07617/) (2022)
- [Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers](https://mlsanity.com/paper/2408.12568/) (2024)
- [A Close Look at Decomposition-based XAI-Methods for Transformer Language Models](https://mlsanity.com/paper/2502.15886/) (2025)
- [Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustness](https://mlsanity.com/paper/2409.01249/) (2024)
- [Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks](https://mlsanity.com/paper/2304.05727/) (2023)
- [Explainable, Domain-Adaptive, and Federated Artificial Intelligence in Medicine](https://mlsanity.com/paper/2211.09317/) (2022)
- [XEdgeAI: A Human-centered Industrial Inspection Framework with Data-centric Explainable Edge AI Approach](https://mlsanity.com/paper/2407.11771/) (2024)
- [LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification](https://mlsanity.com/paper/2204.02965/) (2022)
- [Explain to Not Forget: Defending Against Catastrophic Forgetting with XAI](https://mlsanity.com/paper/2205.01929/) (2022)
- [Toward Compact Deep Neural Networks via Energy-Aware Pruning](https://mlsanity.com/paper/2103.10858/) (2021)
- [Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability](https://mlsanity.com/paper/2108.01335/) (2021)
- [Measurably Stronger Explanation Reliability via Model Canonization](https://mlsanity.com/paper/2202.06621/) (2022)
- [Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs](https://mlsanity.com/paper/2206.14658/) (2022)
- [Less is More: The Influence of Pruning on the Explainability of CNNs](https://mlsanity.com/paper/2302.08878/) (2023)
- [Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy](https://mlsanity.com/paper/2111.09635/) (2021)

## Abstract (arXiv)

The success of convolutional neural networks (CNNs) in various applications is accompanied by a significant increase in computation and parameter storage costs. Recent efforts to reduce these overheads involve pruning and compressing the weights of various layers while at the same time aiming to not sacrifice performance. In this paper, we propose a novel criterion for CNN pruning inspired by neural network interpretability: The most relevant units, i.e. weights or filters, are automatically found using their relevance scores obtained from concepts of explainable AI (XAI). By exploring this idea, we connect the lines of interpretability and model compression research. We show that our proposed method can efficiently prune CNN models in transfer-learning setups in which networks pre-trained on large corpora are adapted to specialized tasks. The method is evaluated on a broad range of computer vision datasets. Notably, our novel criterion is not only competitive or better compared to state-of-the-art pruning criteria when successive retraining is performed, but clearly outperforms these previous criteria in the resource-constrained application scenario in which the data of the task to be transferred to is very scarce and one chooses to refrain from fine-tuning. Our method is able to compress the model iteratively while maintaining or even improving accuracy. At the same time, it has a computational cost in the order of gradient computation and is comparatively simple to apply without the need for tuning hyperparameters for pruning.