# Pruning Early Exit Networks

arXiv: https://arxiv.org/abs/2207.03644 · HTML: https://mlsanity.com/paper/2207.03644/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 8 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Convolutional Networks with Adaptive Inference Graphs](https://mlsanity.com/paper/1711.11503/) (2017)
- [Spatially Adaptive Computation Time for Residual Networks](https://mlsanity.com/paper/1612.02297/) (2016)
- [Fractional Skipping: Towards Finer-Grained Dynamic CNN Inference](https://mlsanity.com/paper/2001.00705/) (2020)
- [E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learning](https://mlsanity.com/paper/2103.01148/) (2021)

## Built on by (most cited first, 4 of 4)

- [Class Based Thresholding in Early Exit Semantic Segmentation Networks](https://mlsanity.com/paper/2210.15621/) (2022)
- [Early-Exit meets Model-Distributed Inference at Edge Networks](https://mlsanity.com/paper/2408.05247/) (2024)
- [Gated Compression Layers for Efficient Always-On Models](https://mlsanity.com/paper/2303.08970/) (2023)
- [Priority-Aware Model-Distributed Inference at Edge Networks](https://mlsanity.com/paper/2412.12371/) (2024)

## Abstract (arXiv)

Deep learning models that perform well often have high computational costs. In this paper, we combine two approaches that try to reduce the computational cost while keeping the model performance high: pruning and early exit networks. We evaluate two approaches of pruning early exit networks: (1) pruning the entire network at once, (2) pruning the base network and additional linear classifiers in an ordered fashion. Experimental results show that pruning the entire network at once is a better strategy in general. However, at high accuracy rates, the two approaches have a similar performance, which implies that the processes of pruning and early exit can be separated without loss of optimality.