# Why should we add early exits to neural networks?

arXiv: https://arxiv.org/abs/2004.12814 · HTML: https://mlsanity.com/paper/2004.12814/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2020.

## Built on by (most cited first, 30 of 70)

- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Universal Model Routing for Efficient LLM Inference](https://mlsanity.com/paper/2502.08773/) (2025)
- [EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelism](https://mlsanity.com/paper/2312.04916/) (2023)
- [When Does Confidence-Based Cascade Deferral Suffice?](https://mlsanity.com/paper/2307.02764/) (2023)
- [Zero Time Waste: Recycling Predictions in Early Exit Neural Networks](https://mlsanity.com/paper/2106.05409/) (2021)
- [Fast yet Safe: Early-Exiting with Risk Control](https://mlsanity.com/paper/2405.20915/) (2024)
- [The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers](https://mlsanity.com/paper/2305.01628/) (2023)
- [Multi-Exit Vision Transformer for Dynamic Inference](https://mlsanity.com/paper/2106.15183/) (2021)
- [ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training](https://mlsanity.com/paper/2110.05323/) (2021)
- [Single-Layer Vision Transformers for More Accurate Early Exits with Less Overhead](https://mlsanity.com/paper/2105.09121/) (2021)
- [Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference](https://mlsanity.com/paper/2206.04685/) (2022)
- [Fixing Overconfidence in Dynamic Neural Networks](https://mlsanity.com/paper/2302.06359/) (2023)
- [Dynamic Split Computing for Efficient Deep Edge Intelligence](https://mlsanity.com/paper/2205.11269/) (2022)
- [HADES: Hardware Accelerated Decoding for Efficient Speculation in Large Language Models](https://mlsanity.com/paper/2412.19925/) (2024)
- [EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models](https://mlsanity.com/paper/2402.00518/) (2024)
- [OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypotheses](https://mlsanity.com/paper/2204.02426/) (2022)
- [Unsupervised Early Exit in DNNs with Multiple Exits](https://mlsanity.com/paper/2209.09480/) (2022)
- [Conditional computation in neural networks: principles and research trends](https://mlsanity.com/paper/2403.07965/) (2024)
- [Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learning](https://mlsanity.com/paper/2104.10461/) (2021)
- [Efficient High-Resolution Deep Learning: A Survey](https://mlsanity.com/paper/2207.13050/) (2022)
- [NACHOS: Neural Architecture Search for Hardware Constrained Early Exit Neural Networks](https://mlsanity.com/paper/2401.13330/) (2024)
- [AHAR: Adaptive CNN for Energy-efficient Human Activity Recognition in Low-power Edge Devices](https://mlsanity.com/paper/2102.01875/) (2021)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Dynamic nsNet2: Efficient Deep Noise Suppression with Early Exiting](https://mlsanity.com/paper/2308.16678/) (2023)
- [Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss](https://mlsanity.com/paper/2102.09808/) (2021)
- [T-RECX: Tiny-Resource Efficient Convolutional neural networks with early-eXit](https://mlsanity.com/paper/2207.06613/) (2022)
- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [Understanding the Robustness of Multi-Exit Models under Common Corruptions](https://mlsanity.com/paper/2212.01562/) (2022)
- [Multi-Exit Kolmogorov-Arnold Networks: enhancing accuracy and parsimony](https://mlsanity.com/paper/2506.03302/) (2025)
- [Adaptive Slimming for Scalable and Efficient Speech Enhancement](https://mlsanity.com/paper/2507.04879/) (2025)

## Abstract (arXiv)

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including: (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.