# Improving Prediction Certainty Estimation for Reliable Early Exiting via Null Space Projection

arXiv: https://arxiv.org/abs/2506.17249 · HTML: https://mlsanity.com/paper/2506.17249/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 8 Jun 2025.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [FastBERT: a Self-distilling BERT with Adaptive Inference Time](https://mlsanity.com/paper/2004.02178/) (2020)
- [The Right Tool for the Job: Matching Model and Instance Complexities](https://mlsanity.com/paper/2004.07453/) (2020)
- [CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade](https://mlsanity.com/paper/2012.14682/) (2020)
- [A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation](https://mlsanity.com/paper/2203.01670/) (2022)
- [F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks](https://mlsanity.com/paper/2305.11916/) (2023)
- [ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference](https://mlsanity.com/paper/2312.11882/) (2023)
- [E-LANG: Energy-Based Joint Inferencing of Super and Swift Language Models](https://mlsanity.com/paper/2203.00748/) (2022)
- [PALBERT: Teaching ALBERT to Ponder](https://mlsanity.com/paper/2204.03276/) (2022)
- [DE3-BERT: Distance-Enhanced Early Exiting for BERT based on Prototypical Networks](https://mlsanity.com/paper/2402.05948/) (2024)
- [COSEE: Consistency-Oriented Signal-Based Early Exiting via Calibrated Sample Weighting Mechanism](https://mlsanity.com/paper/2412.13236/) (2024)

## Built on by (most cited first, 2 of 2)

- [Learning Hierarchical Knowledge in Text-Rich Networks with Taxonomy-Informed Representation Learning](https://mlsanity.com/paper/2603.08159/) (2026)
- [Detecting Miscitation on the Scholarly Web through LLM-Augmented Text-Rich Graph Learning](https://mlsanity.com/paper/2603.12290/) (2026)

## Abstract (arXiv)

Early exiting has demonstrated great potential in accelerating the inference of pre-trained language models (PLMs) by enabling easy samples to exit at shallow layers, eliminating the need for executing deeper layers. However, existing early exiting methods primarily rely on class-relevant logits to formulate their exiting signals for estimating prediction certainty, neglecting the detrimental influence of class-irrelevant information in the features on prediction certainty. This leads to an overestimation of prediction certainty, causing premature exiting of samples with incorrect early predictions. To remedy this, we define an NSP score to estimate prediction certainty by considering the proportion of class-irrelevant information in the features. On this basis, we propose a novel early exiting method based on the Certainty-Aware Probability (CAP) score, which integrates insights from both logits and the NSP score to enhance prediction certainty estimation, thus enabling more reliable exiting decisions. The experimental results on the GLUE benchmark show that our method can achieve an average speed-up ratio of 2.19x across all tasks with negligible performance degradation, surpassing the state-of-the-art (SOTA) ConsistentEE by 28%, yielding a better trade-off between task performance and inference efficiency. The code is available at https://github.com/He-Jianing/NSP.git.