mlsanityData to 7 Oct 2026

Pruning Early Exit Networks

arXiv 2207.03644 ↗First posted 8 Jul 2022cs.LG cs.CV cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

Deep learning models that perform well often have high computational costs. In this paper, we combine two approaches that try to reduce the computational cost while keeping the model performance high: pruning and early exit networks. We evaluate two approaches of pruning early exit networks: (1) pruning the entire network at once, (2) pruning the base network and additional linear classifiers in an ordered fashion. Experimental results show that pruning the entire network at once is a better strategy in general. However, at high accuracy rates, the two approaches have a similar performance, which implies that the processes of pruning and early exit can be separated without loss of optimality.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 13 of its 16 references that are in the map. First posted on arXiv 8 Jul 2022.

Builds on

13 of 16 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  6. 2018Rethinking the Value of Network Pruningcited by 595
  7. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  8. 2020What is the State of Neural Network Pruning?cited by 578
  9. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  10. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  11. 2016Spatially Adaptive Computation Time for Residual Networkscited by 118
  12. 2020Fractional Skipping: Towards Finer-Grained Dynamic CNN Inferencecited by 23
1 more
  1. 2021E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learningcited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2022Class Based Thresholding in Early Exit Semantic Segmentation Networkscited by 5
  2. 2024Early-Exit meets Model-Distributed Inference at Edge Networkscited by 3
  3. 2023Gated Compression Layers for Efficient Always-On Modelscited by 2
  4. 2024Priority-Aware Model-Distributed Inference at Edge Networkscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
  2. 2019Structural Pruning in Deep Neural Networks: A Small-World Approach
  3. 2024Is Oracle Pruning the True Oracle?
  4. 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
  5. 2025A Survey of Early Exit Deep Neural Networks in NLP
  6. 2022QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures
  7. 2026Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit
  8. 2021Only Train Once: A One-Shot Neural Network Training And Pruning Framework