mlsanityData to 7 Oct 2026

Towards Anytime Classification in Early-Exit Architectures by Enforcing Conditional Monotonicity

arXiv 2306.02652 ↗First posted 5 Jun 2023cs.LG cs.AI stat.MLcited by 16

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Modern predictive models are often deployed to environments in which computational budgets are dynamic. Anytime algorithms are well-suited to such environments as, at any point during computation, they can output a prediction whose quality is a function of computation time. Early-exit neural networks have garnered attention in the context of anytime computation due to their capability to provide intermediate predictions at various stages throughout the network. However, we demonstrate that current early-exit networks are not directly applicable to anytime settings, as the quality of predictions for individual data points is not guaranteed to improve with longer computation. To address this shortcoming, we propose an elegant post-hoc modification, based on the Product-of-Experts, that encourages an early-exit network to become gradually confident. This gives our deep models the property of conditional monotonicity in the prediction quality -- an essential stepping stone towards truly anytime predictive modeling using early-exit architectures. Our empirical results on standard image-classification tasks demonstrate that such behaviors can be achieved while preserving competitive accuracy on average.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 42 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

26 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Learning to Weight Samples for Dynamic Early-exiting Networks17 Sep
  2. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networks9 Jun
  3. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges8 Mar

Other references · 23

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  3. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  4. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  5. 2021Dynamic Neural Networks: A Surveycited by 405
  6. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  7. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  8. 2018Slimmable Neural Networkscited by 301
  9. 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
  10. 2020Resolution Adaptive Networks for Efficient Inferencecited by 146
  11. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  12. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 125
11 more
  1. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  2. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  3. 2014Generalized Product of Experts for Automatic and Principled Fusion of Gaussian Process Predictionscited by 66
  4. 2019Improved Techniques for Training Adaptive Deep Networkscited by 66
  5. 2020Depth Uncertainty in Neural Networkscited by 56
  6. 2022A Survey on Dynamic Neural Networks for Natural Language Processingcited by 24
  7. 2023Fixing Overconfidence in Dynamic Neural Networkscited by 19
  8. 2021Anytime Dense Prediction with Confidence Adaptivitycited by 18
  9. 2021Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimationcited by 18
  10. 2020Learning from demonstration using products of experts: applications to manipulation and task prioritizationcited by 4
  11. 2013State-space Abstraction for Anytime Evaluation of Probabilistic Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 4; 2026: 2.

16 papers.

Sort:

By area:

  1. 2024Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensemblingcited by 33
  2. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 27
  3. 2024Cascade-Aware Training of Language Modelscited by 15
  4. 2024On the Role of Depth and Looping for In-Context Learning with Task Diversitycited by 11
  5. 2025Language Models Can Predict Their Own Behaviorcited by 8
  6. 2023Early-Exit Neural Networks with Nested Prediction Setscited by 6
  7. 2024QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoringcited by 4
  8. 2024RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inferencecited by 4
  9. 2024Dynamic Vocabulary Pruning in Early-Exit LLMscited by 4
  10. 2024Performance Control in Early Exiting to Deploy Large Models at the Same Cost of Smaller Onescited by 4
  11. 2025Controlling the Risk of Corrupted Contexts for Language Models via Early-Exitingcited by 3
  12. 2024DuoDiff: Accelerating Diffusion Models with a Dual-Backbone Approachcited by 1
4 more
  1. 2025TCUQ: Single-Pass Uncertainty Quantification from Temporal Consistency with Streaming Conformal Calibration for TinyMLcited by 0
  2. 2025Rethinking Calibration for Early-Exit Neural Networkscited by 0
  3. 2026RRFC: Recursive Refinement via Feedback Conditioning for Iterative Image-to-Image Generationcited by 0
  4. 2026Does per-frame early exit pay? A compute-matched study of dynamic depth for on-device speech enhancementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025You Only Look Once at Anytime (AnytimeYOLO): Analysis and Optimization of Early-Exits for Object-Detection
  2. 2026Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
  3. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directions
  4. 2025Beyond Greedy Exits: Improved Early Exit Decisions for Risk Control and Reliability
  5. 2022Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference
  6. 2023Early-Exit with Class Exclusion for Efficient Inference of Neural Networks
  7. 2022QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures
  8. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Models