# AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability arXiv: https://arxiv.org/abs/2410.18351 ยท HTML: https://mlsanity.com/paper/2410.18351/ ## Key facts - Cited by 25 papers in the map (data through 14 Oct 2026). - Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. - Builds on 16 of its 23 references that are in the map. - First posted on arXiv 24 Oct 2024. ## Placed in - Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/) ## Builds on (most cited first, 16 of 16) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023) - [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022) - [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023) - [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023) - [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022) - [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017) - [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023) - [TinyLlama: An Open-Source Small Language Model](https://mlsanity.com/paper/2401.02385/) (2024) - [Speculative Decoding with Big Little Decoder](https://mlsanity.com/paper/2302.07863/) (2023) - [SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths](https://mlsanity.com/paper/2405.19715/) (2024) - [Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models](https://mlsanity.com/paper/2405.04304/) (2024) - [Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs](https://mlsanity.com/paper/2403.00858/) (2024) - [The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation](https://mlsanity.com/paper/2302.06784/) (2023) ## Built on by (most cited first, 25 of 25) - [TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding](https://mlsanity.com/paper/2502.15197/) (2025) - [Speeding up Speculative Decoding via Sequential Approximate Verification](https://mlsanity.com/paper/2502.04557/) (2025) - [DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding](https://mlsanity.com/paper/2504.05598/) (2025) - [Optimal Stopping vs Best-of-N for Inference Time Optimization](https://mlsanity.com/paper/2510.01394/) (2025) - [VeriCache: Turning Lossy KV Cache into Lossless LLM Inference](https://mlsanity.com/paper/2605.17613/) (2026) - [OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decoding](https://mlsanity.com/paper/2507.02659/) (2025) - [Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding](https://mlsanity.com/paper/2509.24328/) (2025) - [HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding](https://mlsanity.com/paper/2505.13254/) (2025) - [DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving](https://mlsanity.com/paper/2509.01083/) (2025) - [TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding](https://mlsanity.com/paper/2511.02017/) (2025) - [AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference](https://mlsanity.com/paper/2512.11280/) (2025) - [PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length](https://mlsanity.com/paper/2602.01274/) (2026) - [AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices](https://mlsanity.com/paper/2604.25326/) (2026) - [Spectral Logit Sculpting: Adaptive Low-Rank Logit Transformation for Controlled Text Generation](https://mlsanity.com/paper/2509.25204/) (2025) - [Polybasic Speculative Decoding Through a Theoretical Perspective](https://mlsanity.com/paper/2510.26527/) (2025) - [Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation](https://mlsanity.com/paper/2602.01956/) (2026) - [TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference](https://mlsanity.com/paper/2602.05145/) (2026) - [MineDraft: A Framework for Batch Parallel Speculative Decoding](https://mlsanity.com/paper/2603.18016/) (2026) - [Adversarial Prompts for Acceptance Collapse in Speculative Decoding](https://mlsanity.com/paper/2607.21804/) (2026) - [From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decoding](https://mlsanity.com/paper/2608.14787/) (2026) - [Vision Is Not Overhead: One-Pass Block Drafting for Lossless Speculative Decoding in Vision-Language Models](https://mlsanity.com/paper/2609.00355/) (2026) - [DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification](https://mlsanity.com/paper/2609.37532/) (2026) - [AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines](https://mlsanity.com/paper/2610.01108/) (2026) - [DLoop: Looped Speculative Decoding](https://mlsanity.com/paper/2610.07659/) (2026) - [SP2ec: Adaptive Self-Speculative Decoding for Vision-Language Models](https://mlsanity.com/paper/2610.18735/) (2026) ## Abstract (arXiv) Speculative decoding is a powerful technique that attempts to circumvent the autoregressive constraint of modern Large Language Models (LLMs). The aim of speculative decoding techniques is to improve the average inference time of a large, target model without sacrificing its accuracy, by using a more efficient draft model to propose draft tokens which are then verified in parallel. The number of draft tokens produced in each drafting round is referred to as the draft length and is often a static hyperparameter chosen based on the acceptance rate statistics of the draft tokens. However, setting a static draft length can negatively impact performance, especially in scenarios where drafting is expensive and there is a high variance in the number of tokens accepted. Adaptive Entropy-based Draft Length (AdaEDL) is a simple, training and parameter-free criteria which allows for early stopping of the token drafting process by approximating a lower bound on the expected acceptance probability of the drafted token based on the currently observed entropy of the drafted logits. We show that AdaEDL consistently outperforms static draft-length speculative decoding by 10%-57% as well as other training-free draft-stopping techniques by upto 10% in a variety of settings and datasets. At the same time, we show that AdaEDL is more robust than these techniques and preserves performance in high-sampling-temperature scenarios. Since it is training-free, in contrast to techniques that rely on the training of dataset-specific draft-stopping predictors, AdaEDL can seamlessly be integrated into a variety of pre-existing LLM systems.