# Resource-Constrained Edge AI with Early Exit Prediction

arXiv: https://arxiv.org/abs/2206.07269 · HTML: https://mlsanity.com/paper/2206.07269/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing](https://mlsanity.com/paper/1910.05316/) (2019)
- [Distributed Deep Neural Networks over the Cloud, the Edge and End Devices](https://mlsanity.com/paper/1709.01921/) (2017)
- [Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges](https://mlsanity.com/paper/2103.04505/) (2021)
- [SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloud](https://mlsanity.com/paper/2008.06402/) (2020)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Communication-Efficient Edge AI: Algorithms and Systems](https://mlsanity.com/paper/2002.09668/) (2020)
- [BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems](https://mlsanity.com/paper/1910.14315/) (2019)
- [Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions](https://mlsanity.com/paper/2106.05022/) (2021)
- [Communication-Computation Trade-Off in Resource-Constrained Edge Inference](https://mlsanity.com/paper/2006.02166/) (2020)
- [Deep feature compression for collaborative object detection](https://mlsanity.com/paper/1802.03931/) (2018)
- [Zero Time Waste: Recycling Predictions in Early Exit Neural Networks](https://mlsanity.com/paper/2106.05409/) (2021)
- [Mobile Edge Intelligence and Computing for the Internet of Vehicles](https://mlsanity.com/paper/1906.00400/) (2019)
- [Edge Intelligence for Autonomous Driving in 6G Wireless System: Design Challenges and Solutions](https://mlsanity.com/paper/2012.06992/) (2020)
- [Branchy-GNN: a Device-Edge Co-Inference Framework for Efficient Point Cloud Processing](https://mlsanity.com/paper/2011.02422/) (2020)
- [Auto-tuning Neural Network Quantization Framework for Collaborative Inference Between the Cloud and Edge](https://mlsanity.com/paper/1812.06426/) (2018)
- [Communication-Computation Efficient Device-Edge Co-Inference via AutoML](https://mlsanity.com/paper/2108.13009/) (2021)

## Built on by (most cited first, 11 of 11)

- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [FIANCEE: Faster Inference of Adversarial Networks via Conditional Early Exits](https://mlsanity.com/paper/2304.10306/) (2023)
- [Recall: Empowering Multimodal Embedding for Edge Devices](https://mlsanity.com/paper/2409.15342/) (2024)
- [Edge Artificial Intelligence: A Systematic Review of Evolution, Taxonomic Frameworks, and Future Horizons](https://mlsanity.com/paper/2510.01439/) (2025)
- [Onboard Optimization and Learning: A Survey](https://mlsanity.com/paper/2505.08793/) (2025)
- [Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits](https://mlsanity.com/paper/2512.20755/) (2025)
- [SINF: Semantic Neural Network Inference with Semantic Subgraphs](https://mlsanity.com/paper/2310.01259/) (2023)
- [Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States](https://mlsanity.com/paper/2603.03146/) (2026)
- [Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks](https://mlsanity.com/paper/2605.05358/) (2026)
- [UAV-Assisted Cooperative Edge Inference for Low-Altitude Economy via MoE-based Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/2605.19290/) (2026)
- [FrugalSOT - Frugal Search Over the Models](https://mlsanity.com/paper/2608.21621/) (2026)

## Abstract (arXiv)

By leveraging the data sample diversity, the early-exit network recently emerges as a prominent neural network architecture to accelerate the deep learning inference process. However, intermediate classifiers of the early exits introduce additional computation overhead, which is unfavorable for resource-constrained edge artificial intelligence (AI). In this paper, we propose an early exit prediction mechanism to reduce the on-device computation overhead in a device-edge co-inference system supported by early-exit networks. Specifically, we design a low-complexity module, namely the Exit Predictor, to guide some distinctly "hard" samples to bypass the computation of the early exits. Besides, considering the varying communication bandwidth, we extend the early exit prediction mechanism for latency-aware edge inference, which adapts the prediction thresholds of the Exit Predictor and the confidence thresholds of the early-exit network via a few simple regression models. Extensive experiment results demonstrate the effectiveness of the Exit Predictor in achieving a better tradeoff between accuracy and on-device computation overhead for early-exit networks. Besides, compared with the baseline methods, the proposed method for latency-aware edge inference attains higher inference accuracy under different bandwidth conditions.