# ADEPT: Adaptive Dynamic Early-Exit Process for Transformers

arXiv: https://arxiv.org/abs/2601.03700 · HTML: https://mlsanity.com/paper/2601.03700/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving.
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 7 Jan 2026.

## Placed in

- Efficient LLM inference and serving / [Early exit and edge inference](https://mlsanity.com/inference/early-exit-and-edge-inference/)

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework](https://mlsanity.com/paper/2202.03052/) (2022)
- [BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks](https://mlsanity.com/paper/1709.01686/) (2017)
- [DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://mlsanity.com/paper/2004.12993/) (2020)
- [Confident Adaptive Language Modeling](https://mlsanity.com/paper/2207.07061/) (2022)
- [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013)
- [BERT Loses Patience: Fast and Robust Inference with Early Exit](https://mlsanity.com/paper/2006.04152/) (2020)
- [Visual Entailment: A Novel Task for Fine-Grained Image Understanding](https://mlsanity.com/paper/1901.06706/) (2019)
- [The emergence of clusters in self-attention dynamics](https://mlsanity.com/paper/2305.05465/) (2023)
- [Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding](https://mlsanity.com/paper/2310.05424/) (2023)
- [Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions](https://mlsanity.com/paper/2305.15083/) (2023)
- [You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model](https://mlsanity.com/paper/2211.11152/) (2022)
- [Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers](https://mlsanity.com/paper/2109.08406/) (2021)
- [Learning to Transfer Prompts for Text Generation](https://mlsanity.com/paper/2205.01543/) (2022)
- [Fine-tuning Large Language Models for Adaptive Machine Translation](https://mlsanity.com/paper/2312.12740/) (2023)
- [F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks](https://mlsanity.com/paper/2305.11916/) (2023)
- [Stage-wise Fine-tuning for Graph-to-Text Generation](https://mlsanity.com/paper/2105.08021/) (2021)
- [BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews](https://mlsanity.com/paper/2107.06802/) (2021)

## Built on by (most cited first, 4 of 4)

- [Tiny Autoregressive Recursive Models](https://mlsanity.com/paper/2603.08082/) (2026)
- [Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching](https://mlsanity.com/paper/2608.09444/) (2026)
- [TIDE: Token-Informed Depth Execution for Per-Token Early Exit in LLM Inference](https://mlsanity.com/paper/2603.21365/) (2026)
- [The Context-Ready Transformer](https://mlsanity.com/paper/2606.27538/) (2026)

## Abstract (arXiv)

The inference of large language models imposes significant computational workloads, often requiring the processing of billions of parameters. Although early-exit strategies have proven effective in reducing computational demands by halting inference earlier, they apply either to only the first token in the generation phase or at the prompt level in the prefill phase. Thus, the Key-Value (KV) cache for skipped layers remains a bottleneck for subsequent token generation, limiting the benefits of early exit. We introduce ADEPT (Adaptive Dynamic Early-exit Process for Transformers), a novel approach designed to overcome this issue and enable dynamic early exit in both the prefill and generation phases. The proposed adaptive token-level early-exit mechanism adjusts computation dynamically based on token complexity, optimizing efficiency without compromising performance. ADEPT further enhances KV generation procedure by decoupling sequential dependencies in skipped layers, making token-level early exit more practical. Experimental results demonstrate that ADEPT improves efficiency by up to 25% in language generation tasks and achieves a 4x speed-up in downstream classification tasks, with up to a 45% improvement in performance.