# Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation

arXiv: https://arxiv.org/abs/2411.18462 · HTML: https://mlsanity.com/paper/2411.18462/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 20 of 20)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://mlsanity.com/paper/2406.20094/) (2024)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)
- [Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding](https://mlsanity.com/paper/2401.07851/) (2024)
- [PEARL: Parallel Speculative Decoding with Adaptive Draft Length](https://mlsanity.com/paper/2408.11850/) (2024)
- [GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding](https://mlsanity.com/paper/2402.02082/) (2024)
- [SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths](https://mlsanity.com/paper/2405.19715/) (2024)
- [PaSS: Parallel Speculative Sampling](https://mlsanity.com/paper/2311.13581/) (2023)
- [A Theoretical Perspective for Speculative Decoding Algorithm](https://mlsanity.com/paper/2411.00841/) (2024)
- [Dynamic Depth Decoding: Faster Speculative Decoding for LLMs](https://mlsanity.com/paper/2409.00142/) (2024)
- [QSpec: Speculative Decoding with Complementary Quantization Schemes](https://mlsanity.com/paper/2410.11305/) (2024)
- [Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation](https://mlsanity.com/paper/2410.10141/) (2024)

## Built on by (most cited first, 16 of 16)

- [TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding](https://mlsanity.com/paper/2606.00487/) (2026)
- [MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification](https://mlsanity.com/paper/2601.15498/) (2026)
- [PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding](https://mlsanity.com/paper/2605.13319/) (2026)
- [Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding](https://mlsanity.com/paper/2509.24328/) (2025)
- [DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Serving](https://mlsanity.com/paper/2509.01083/) (2025)
- [TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding](https://mlsanity.com/paper/2511.02017/) (2025)
- [PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length](https://mlsanity.com/paper/2602.01274/) (2026)
- [TIDE: Temporal Incremental Draft Engine for Self-Improving LLM Inference](https://mlsanity.com/paper/2602.05145/) (2026)
- [SMART: When is it Actually Worth Expanding a Speculative Tree?](https://mlsanity.com/paper/2604.09731/) (2026)
- [FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving](https://mlsanity.com/paper/2604.20503/) (2026)
- [A Sparse Glimpse of the Whole: Train-Free Self-Speculative Decoding](https://mlsanity.com/paper/2607.27735/) (2026)
- [The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts](https://mlsanity.com/paper/2609.22156/) (2026)
- [MemSpec: Memory-Aware Runtime for Adaptive Draft Scheduling in Speculative Decoding on Edge Devices](https://mlsanity.com/paper/2608.10362/) (2026)
- [AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines](https://mlsanity.com/paper/2610.01108/) (2026)
- [DLoop: Looped Speculative Decoding](https://mlsanity.com/paper/2610.07659/) (2026)
- [SP2ec: Adaptive Self-Speculative Decoding for Vision-Language Models](https://mlsanity.com/paper/2610.18735/) (2026)

## Abstract (arXiv)

Conventional speculative decoding (SD) methods utilize a predefined length policy for proposing drafts, which implies the premise that the target model smoothly accepts the proposed draft tokens. However, reality deviates from this assumption: the oracle draft length varies significantly, and the fixed-length policy hardly satisfies such a requirement. Moreover, such discrepancy is further exacerbated in scenarios involving complex reasoning and long-form generation, particularly under test-time scaling for reasoning-specialized models. Through both theoretical and empirical estimation, we establish that the discrepancy between the draft and target models can be approximated by the draft model's prediction entropy: a high entropy indicates a low acceptance rate of draft tokens, and vice versa. Based on this insight, we propose SVIP: Self-Verification Length Policy for Long-Context Speculative Decoding, which is a training-free dynamic length policy for speculative decoding systems that adaptively determines the lengths of draft sequences by referring to the draft entropy. Experimental results on mainstream SD benchmarks as well as reasoning-heavy benchmarks demonstrate the superior performance of SVIP, achieving up to 17% speedup on MT-Bench at 8K context compared with fixed draft lengths, and 22% speedup for QwQ in long-form reasoning.