# Early Exit Is a Natural Capability in Transformer-based Models: An Empirical Study on Early Exit without Joint Optimization

arXiv: https://arxiv.org/abs/2412.01455 · HTML: https://mlsanity.com/paper/2412.01455/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 2 Dec 2024.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Built on by (most cited first, 9 of 9)

- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive Depth](https://mlsanity.com/paper/2603.01914/) (2026)
- [Entropy-Lens: Uncovering Decision Strategies in LLMs](https://mlsanity.com/paper/2502.16570/) (2025)
- [Amnesia: Adversarial Semantic Layer Specific Activation Steering in Large Language Models](https://mlsanity.com/paper/2603.10080/) (2026)
- [TIDE: Token-Informed Depth Execution for Per-Token Early Exit in LLM Inference](https://mlsanity.com/paper/2603.21365/) (2026)
- [Depth Adaptive Efficient Visual Autoregressive Modeling](https://mlsanity.com/paper/2604.17286/) (2026)
- [N-vium: Mixture-of-Exits Transformer for Accelerated Exact Generation](https://mlsanity.com/paper/2605.13190/) (2026)
- [HALO: Hybrid Adaptive Latent Reasoning for Language Models](https://mlsanity.com/paper/2607.08775/) (2026)
- [When Less Compute Is More: Adaptive Early Exit Improves Pretrained Outlier Detection](https://mlsanity.com/paper/2609.32898/) (2026)

## Abstract (arXiv)

Large language models (LLMs) exhibit exceptional performance across various downstream tasks. However, they encounter limitations due to slow inference speeds stemming from their extensive parameters. The early exit (EE) is an approach that aims to accelerate auto-regressive decoding. EE generates outputs from intermediate layers instead of using the whole model, which offers a promising solution to this challenge. However, additional output layers and joint optimization used in conventional EE hinder the application of EE in LLMs. In this paper, we explore the possibility of LLMs EE without additional output layers and joint optimization. Our findings indicate that EE is a natural capability within transformer-based models. While joint optimization does not give model EE capability, it must be employed to address challenges by improving the accuracy of locating the optimal EE layer through gating functions. Additionally, our study reveals patterns in EE behavior from a sub-word perspective based on the LLaMA model and the potential possibility for EE based on sub-layers.