mlsanityData to 14 Oct 2026

COSEE: Consistency-Oriented Signal-Based Early Exiting via Calibrated Sample Weighting Mechanism

arXiv 2412.13236 ↗First posted 17 Dec 2024cs.LG cs.AIcited by 1

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Early exiting is an effective paradigm for improving the inference efficiency of pre-trained language models (PLMs) by dynamically adjusting the number of executed layers for each sample. However, in most existing works, easy and hard samples are treated equally by each classifier during training, which neglects the test-time early exiting behavior, leading to inconsistency between training and testing. Although some methods have tackled this issue under a fixed speed-up ratio, the challenge of flexibly adjusting the speed-up ratio while maintaining consistency between training and testing is still under-explored. To bridge the gap, we propose a novel Consistency-Oriented Signal-based Early Exiting (COSEE) framework, which leverages a calibrated sample weighting mechanism to enable each classifier to emphasize the samples that are more likely to exit at that classifier under various acceleration scenarios. Extensive experiments on the GLUE benchmark demonstrate the effectiveness of our COSEE across multiple exiting signals and backbones, yielding a better trade-off between performance and efficiency.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 11 of its 24 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

11 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference19 Dec
  2. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation3 Mar

Other references · 9

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  5. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 332
  6. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 226
  7. 2020The Right Tool for the Job: Matching Model and Instance Complexitiescited by 126
  8. 2020CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascadecited by 47
  9. 2023F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Taskscited by 18

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Improving Prediction Certainty Estimation for Reliable Early Exiting via Null Space Projectioncited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024An Efficient Inference Framework for Early-exit Large Language Models
  2. 2026The Diminishing Returns of Early-Exit Decoding in Modern LLMs
  3. 2024RAEE: A Robust Retrieval-Augmented Early Exit Framework for Efficient Inference
  4. 2024Dynamic Vocabulary Pruning in Early-Exit LLMs
  5. 2024EE-Tuning: An Economical yet Scalable Solution for Tuning Early-Exit Large Language Models
  6. 2026ADEPT: Adaptive Dynamic Early-Exit Process for Transformers
  7. 2026Balancing Stability and Plasticity in Sequentially Trained Early-Exiting Neural Networks
  8. 2025Rethinking Calibration for Early-Exit Neural Networks