mlsanityData to 7 Oct 2026

Fast yet Safe: Early-Exiting with Risk Control

arXiv 2405.20915 ↗First posted 31 May 2024cs.LG cs.AI cs.CVcited by 27

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Scaling machine learning models significantly improves their performance. However, such gains come at the cost of inference being slow and resource-intensive. Early-exit neural networks (EENNs) offer a promising solution: they accelerate inference by allowing intermediate layers to exit and produce a prediction early. Yet a fundamental issue with EENNs is how to determine when to exit without severely degrading performance. In other words, when is it 'safe' for an EENN to go 'fast'? To address this issue, we investigate how to adapt frameworks of risk control to EENNs. Risk control offers a distribution-free, post-hoc solution that tunes the EENN's exiting mechanism so that exits only occur when the output is of sufficient quality. We empirically validate our insights on a range of vision and language tasks, demonstrating that risk control can produce substantial computational savings, all the while preserving user-specified performance goals.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 61 of its 90 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

61 of 90 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models1 Feb
  2. 2023Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity5 Jun
  3. 2022Learning to Weight Samples for Dynamic Early-exiting Networks17 Sep
  4. 2022Confident Adaptive Language Modeling14 Jul
  5. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networks9 Jun

Other references · 56

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  9. 2015Teaching Machines to Read and Comprehendcited by 1,039
  10. 2016Playing for Data: Ground Truth from Computer Gamescited by 816
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  12. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
38 more
  1. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  3. 2021Dynamic Neural Networks: A Surveycited by 405
  4. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  5. 2022Conformal Risk Controlcited by 290
  6. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  7. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  8. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  9. 2019Depth-Adaptive Transformercited by 227
  10. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  11. 2020Estimating means of bounded random variables by bettingcited by 164
  12. 2023Speculative Decoding with Big Little Decodercited by 163
  13. 2023Conformal Language Modelingcited by 163
  14. 2017Adaptive Neural Networks for Efficient Inferencecited by 161
  15. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  16. 2020Conformal Prediction: a Unified Review of Theory and New Challengescited by 118
  17. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  18. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  19. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  20. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  21. 2020Why should we add early exits to neural networks?cited by 70
  22. 2024Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Modelscited by 64
  23. 2021Sensitivity Analysis of Individual Treatment Effects: A Robust Conformal Inference Approachcited by 62
  24. 2019PAC Confidence Sets for Deep Neural Networks via Calibrated Predictioncited by 59
  25. 2023How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Controlcited by 50
  26. 2022Achieving Risk Control in Online Learning Settingscited by 38
  27. 2023Dynamic Perceiver for Efficient Visual Recognitioncited by 33
  28. 2020Learning to Stop While Learning to Predictcited by 30
  29. 2023Principal Uncertainty Quantification with Spatial Correlation for Image Restoration Problemscited by 28
  30. 2022A Survey on Dynamic Neural Networks for Natural Language Processingcited by 24
  31. 2024Adaptive Bounding Box Uncertainties via Two-Step Conformal Predictioncited by 23
  32. 2022Semantic uncertainty intervals for disentangled latent spacescited by 22
  33. 2022Efficiently Controlling Multiple Risks with Pareto Testingcited by 21
  34. 2023Fixing Overconfidence in Dynamic Neural Networkscited by 19
  35. 2021Anytime Dense Prediction with Confidence Adaptivitycited by 18
  36. 2023Prompt Risk Control: A Rigorous Framework for Responsible Deployment of Large Language Modelscited by 17
  37. 2023Jointly-Learned Exit and Inference for a Dynamic Neural Network : JEI-DNNcited by 15
  38. 2022Quantile Risk Control: A Flexible Framework for Bounding the Probability of High-Loss Predictionscited by 14

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 16.

27 papers.

Sort:

By area:

  1. 2025Adaptive Layer-skipping in Pre-trained LLMscited by 24
  2. 2025Entropy After </Think> for reasoning model early exitingcited by 24
  3. 2024Semi-Supervised Risk Control via Prediction-Powered Inferencecited by 13
  4. 2026Conformal Thinking: Risk Control for Reasoning on a Compute Budgetcited by 9
  5. 2024Dynamic Vocabulary Pruning in Early-Exit LLMscited by 4
  6. 2025Statistical Methods in Generative AIcited by 4
  7. 2025Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliabilitycited by 3
  8. 2025Controlling the Risk of Corrupted Contexts for Language Models via Early-Exitingcited by 3
  9. 2025Knowing When to Quit: Probabilistic Early Exits for Speech Separationcited by 2
  10. 2024Confident magnitude-based neural network pruningcited by 1
  11. 2026Temporal Difference Calibration in Sequential Tasks: Application to Vision-Language-Action Modelscited by 1
  12. 2026MACRO: Markov Chain Routing of Transformer Layerscited by 1
15 more
  1. 2026Revelation Controlcited by 1
  2. 2025SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Tokencited by 0
  3. 2025Rethinking Calibration for Early-Exit Neural Networkscited by 0
  4. 2026Sequence Repetition Enhances Token Embeddings and Improves Sequence Labeling with Decoder-only Language Modelscited by 0
  5. 2026When Do Early-Exit Networks Generalize? A PAC-Bayesian Theory of Adaptive Depthcited by 0
  6. 2026A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networkscited by 0
  7. 2026SAND: Spatially Adaptive Network Depth for Fast Sampling of Neural Implicit Surfacescited by 0
  8. 2026Amortized-Precision Quantization for Early-Exit Vision Transformerscited by 0
  9. 2026End-to-End Dynamic Sparsity for Resource-Adaptive LLM Inferencecited by 0
  10. 2026Online Safety Monitoring for LLMscited by 0
  11. 2026TRACE-CRC: Trajectory-Adaptive Conformal Risk Control for Multi-Step Channel State Information Predictioncited by 0
  12. 2026Occupancy-based Quantile Risk Controlcited by 0
  13. 2026Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exitcited by 0
  14. 2026Prediction Dynamics in Depth-Recurrent Language Modelscited by 0
  15. 2026Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targetscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Ones
  2. 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
  3. 2025Know What You Don't Know: Selective Prediction for Early Exit DNNs
  4. 2023Adaptive Deep Neural Network Inference Optimization with EENet
  5. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exits
  6. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
  7. 2025A Survey of Early Exit Deep Neural Networks in NLP
  8. 2026Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks