# Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference

arXiv: https://arxiv.org/abs/2206.04685 · HTML: https://mlsanity.com/paper/2206.04685/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [SkipNet: Learning Dynamic Routing in Convolutional Networks](https://mlsanity.com/paper/1711.09485/) (2017)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [Distributed Deep Neural Networks over the Cloud, the Edge and End Devices](https://mlsanity.com/paper/1709.01921/) (2017)
- [Spatially Adaptive Computation Time for Residual Networks](https://mlsanity.com/paper/1612.02297/) (2016)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [Why should we add early exits to neural networks?](https://mlsanity.com/paper/2004.12814/) (2020)
- [EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference](https://mlsanity.com/paper/2011.14203/) (2020)
- [Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognition](https://mlsanity.com/paper/1509.08971/) (2015)
- [FrameExit: Conditional Early Exiting for Efficient Video Recognition](https://mlsanity.com/paper/2104.13400/) (2021)
- [Zero Time Waste: Recycling Predictions in Early Exit Neural Networks](https://mlsanity.com/paper/2106.05409/) (2021)

## Built on by (most cited first, 24 of 24)

- [Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNN](https://mlsanity.com/paper/2310.09163/) (2023)
- [HADAS: Hardware-Aware Dynamic Neural Architecture Search for Edge Performance Scaling](https://mlsanity.com/paper/2212.03354/) (2022)
- [DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference](https://mlsanity.com/paper/2306.01811/) (2023)
- [HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving](https://mlsanity.com/paper/2504.10724/) (2025)
- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [FIANCEE: Faster Inference of Adversarial Networks via Conditional Early Exits](https://mlsanity.com/paper/2304.10306/) (2023)
- [Recall: Empowering Multimodal Embedding for Edge Devices](https://mlsanity.com/paper/2409.15342/) (2024)
- [Energy-Aware Deep Learning on Resource-Constrained Hardware](https://mlsanity.com/paper/2505.12523/) (2025)
- [AI Model Modulation with Logits Redistribution](https://mlsanity.com/paper/2603.12755/) (2026)
- [EERO: Early Exit with Reject Option for Efficient Classification with limited budget](https://mlsanity.com/paper/2402.03779/) (2024)
- [Sensor-Aware Classifiers for Energy-Efficient Time Series Applications on IoT Devices](https://mlsanity.com/paper/2407.08715/) (2024)
- [E4: Energy-Efficient DNN Inference for Edge Video Analytics Via Early-Exit and DVFS](https://mlsanity.com/paper/2503.04865/) (2025)
- [Order of Compression: A Systematic and Optimal Sequence to Combinationally Compress CNN](https://mlsanity.com/paper/2403.17447/) (2024)
- [Don't Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNs](https://mlsanity.com/paper/2408.03223/) (2024)
- [M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference](https://mlsanity.com/paper/2502.02040/) (2025)
- [Onboard Optimization and Learning: A Survey](https://mlsanity.com/paper/2505.08793/) (2025)
- [River-LLM: Large Language Model Seamless Exit Based on KV Share](https://mlsanity.com/paper/2604.18396/) (2026)
- [NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inference](https://mlsanity.com/paper/2305.14405/) (2023)
- [Lightweight Remote Sensing Scene Classification on Edge Devices via Knowledge Distillation and Early-exit](https://mlsanity.com/paper/2507.20623/) (2025)
- [CalexNet: Soft Cascade-Aligned Training and Calibration for Lightweight Early-Exit Branches](https://mlsanity.com/paper/2509.08318/) (2025)
- [ERDE: Entropy-Regularized Distillation for Early-exit](https://mlsanity.com/paper/2510.04856/) (2025)
- [Efficient Deep Learning for Biometrics: Overview, Challenges and Trends in Ear of Frugal AI](https://mlsanity.com/paper/2602.08809/) (2026)
- [Two-dimensional early exit optimisation of LLM inference](https://mlsanity.com/paper/2604.18592/) (2026)
- [Adaptive AI: Energy Efficient Multi-exit TinyML on Intelligent Vision Systems at the Edge](https://mlsanity.com/paper/2609.11939/) (2026)

## Abstract (arXiv)

By adding exiting layers to the deep learning networks, early exit can terminate the inference earlier with accurate results. The passive decision-making of whether to exit or continue the next layer has to go through every pre-placed exiting layer until it exits. In addition, it is also hard to adjust the configurations of the computing platforms alongside the inference proceeds. By incorporating a low-cost prediction engine, we propose a Predictive Exit framework for computation- and energy-efficient deep learning applications. Predictive Exit can forecast where the network will exit (i.e., establish the number of remaining layers to finish the inference), which effectively reduces the network computation cost by exiting on time without running every pre-placed exiting layer. Moreover, according to the number of remaining layers, proper computing configurations (i.e., frequency and voltage) are selected to execute the network to further save energy. Extensive experimental results demonstrate that Predictive Exit achieves up to 96.2% computation reduction and 72.9% energy-saving compared with classic deep learning networks; and 12.8% computation reduction and 37.6% energy-saving compared with the early exit under state-of-the-art exiting strategies, given the same inference accuracy and latency.