mlsanityData to 14 Oct 2026

Rethinking Calibration for Early-Exit Neural Networks

arXiv 2508.21495 ↗First posted 29 Aug 2025cs.LGcited by 0

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enough. Most methods rely on confidence thresholds for exiting, and consequently, improving classifier calibration is widely assumed to improve performance. In this work, we challenge this assumption and show that calibration alone is not sufficient for EENNs to exploit adaptive computation. To address this insufficiency, we introduce Early-Exit Failure Prediction (EEFP), which accounts for both prediction correctness and the cost of further computation. We also propose a lightweight, EEFP-motivated procedure to improve the intermediate classifiers, which can directly replace calibration in EENNs. Extensive experiments demonstrate that our approach achieves superior cost-accuracy trade-offs compared to calibration, and EEFP more reliably reflects overall EENN performance. Our code is available at https://github.com/gmum/rethinking-calibration-for-eenns.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 33 of its 57 references that are in the map. First posted on arXiv 29 Aug 2025.

Builds on

33 of 57 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones26 Dec
  2. 2024Fast yet Safe: Early-Exiting with Risk Control31 May
  3. 2023Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity5 Jun
  4. 2022Confident Adaptive Language Modeling14 Jul

Other references · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  8. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  9. 2014Deeply-Supervised Netscited by 367
  10. 2019Verified Uncertainty Calibrationcited by 309
  11. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  12. 2020Consistent Estimators for Learning to Defer to an Expertcited by 271
17 more
  1. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  2. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  3. 2019Addressing Failure Prediction by Learning Model Confidencecited by 186
  4. 2019Evaluating model calibration in classificationcited by 168
  5. 2020SPINN: Synergistic Progressive Inference of Neural Networks over Device and Cloudcited by 112
  6. 2020Learning Dynamic Routing for Semantic Segmentationcited by 79
  7. 2020EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencecited by 70
  8. 2015Conditional Deep Learning for Energy-Efficient and Enhanced Pattern Recognitioncited by 67
  9. 2019Improved Techniques for Training Adaptive Deep Networkscited by 66
  10. 2022Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensemblescited by 60
  11. 2023When Does Confidence-Based Cascade Deferral Suffice?cited by 60
  12. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  13. 2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
  14. 2020Branchy-GNN: a Device-Edge Co-Inference Framework for Efficient Point Cloud Processingcited by 21
  15. 2023Fixing Overconfidence in Dynamic Neural Networkscited by 19
  16. 2023Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNNcited by 15
  17. 2023Early-Exit Neural Networks with Nested Prediction Setscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
  2. 2023Adaptive Deep Neural Network Inference Optimization with EENet
  3. 2021E2CM: Early Exit via Class Means for Efficient Supervised and Unsupervised Learning
  4. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
  5. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits
  6. 2025Improving Prediction Certainty Estimation for Reliable Early Exiting via Null Space Projection
  7. 2023Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings
  8. 2025CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment