# On Causal Inference for Data-free Structured Pruning

arXiv: https://arxiv.org/abs/2112.10229 · HTML: https://mlsanity.com/paper/2112.10229/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 15 references that are in the map.
- First posted on arXiv 19 Dec 2021.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Data-free parameter pruning for Deep Neural Networks](https://mlsanity.com/paper/1507.06149/) (2015)
- [Data-Independent Neural Pruning via Coresets](https://mlsanity.com/paper/1907.04018/) (2019)
- [An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms](https://mlsanity.com/paper/1909.06727/) (2019)
- [Explaining Deep Learning Models using Causal Inference](https://mlsanity.com/paper/1811.04376/) (2018)
- [COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning](https://mlsanity.com/paper/1906.10337/) (2019)
- [MINT: Deep Network Compression via Mutual Information-based Neuron Trimming](https://mlsanity.com/paper/2003.08472/) (2020)
- [Deeplite Neutrino: An End-to-End Framework for Constrained Deep Learning Model Optimization](https://mlsanity.com/paper/2101.04073/) (2021)
- [Examining the causal structures of deep neural networks using information theory](https://mlsanity.com/paper/2010.13871/) (2020)

## Abstract (arXiv)

Neural networks (NNs) are making a large impact both on research and industry. Nevertheless, as NNs' accuracy increases, it is followed by an expansion in their size, required number of compute operations and energy consumption. Increase in resource consumption results in NNs' reduced adoption rate and real-world deployment impracticality. Therefore, NNs need to be compressed to make them available to a wider audience and at the same time decrease their runtime costs. In this work, we approach this challenge from a causal inference perspective, and we propose a scoring mechanism to facilitate structured pruning of NNs. The approach is based on measuring mutual information under a maximum entropy perturbation, sequentially propagated through the NN. We demonstrate the method's performance on two datasets and various NNs' sizes, and we show that our approach achieves competitive performance under challenging conditions.