mlsanityData to 7 Oct 2026

Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models

arXiv 2405.04304 ↗First posted 7 May 2024cs.CLcited by 29

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding is commonly used for reducing the inference latency of large language models. Its effectiveness depends highly on the speculation lookahead (SL)-the number of tokens generated by the draft model at each iteration. In this work we show that the common practice of using the same SL for all iterations (static SL) is suboptimal. We introduce DISCO (DynamIc SpeCulation lookahead Optimization), a novel method for dynamically selecting the SL. Our experiments with four datasets show that DISCO reaches an average speedup of 10% compared to the best static SL baseline, while generating the exact same text.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 25 references that are in the map. First posted on arXiv 7 May 2024.

Builds on

12 of 25 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Speculative Streaming: Fast LLM Inference without Auxiliary Models16 Feb
  2. 2024Tandem Transformers for Inference Efficient LLMs13 Feb
  3. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  4. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  5. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 6

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 809
  5. 2023StarCoder: may the source be with you!cited by 790
  6. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 17.

29 papers.

Sort:

By area:

  1. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengthscited by 73
  2. 2024AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probabilitycited by 24
  3. 2025Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabulariescited by 20
  4. 2024AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structurescited by 17
  5. 2026LLM-42: Enabling Determinism in LLM Inference with Verified Speculationcited by 14
  6. 2025Speeding up Speculative Decoding via Sequential Approximate Verificationcited by 11
  7. 2025MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Expertscited by 11
  8. 2024Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inferencecited by 9
  9. 2025AutoChemSchematic AI: Agentic Physics-Aware Automation for Chemical Manufacturing Scale-Upcited by 6
  10. 2026DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verificationcited by 5
  11. 2026Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decodingcited by 5
  12. 2025Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMscited by 4
17 more
  1. 2025OmniDraft: A Cross-vocabulary, Online Adaptive Drafter for On-device Speculative Decodingcited by 4
  2. 2026D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decodingcited by 4
  3. 2026AngelSpec: Towards Real-World High Performance Inference with Speculative Decodingcited by 4
  4. 2026PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Lengthcited by 2
  5. 2026TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rolloutscited by 2
  6. 2025DSDE: Dynamic Speculative Decoding with KLD Stability for Real-World Servingcited by 1
  7. 2026SMART: When is it Actually Worth Expanding a Speculative Tree?cited by 1
  8. 2026Margins, Not Windows: Training-Free Per-Step Lossy Speculative Decodingcited by 1
  9. 2025Automatic Task Detection and Heterogeneous LLM Speculative Decodingcited by 0
  10. 2026Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowingcited by 0
  11. 2026An Interpretable Latency Model for Speculative Decoding in LLM Servingcited by 0
  12. 2026Fully Rotation-Equivariant Spectral-Spatial Learning for Multispectral Object Detectioncited by 0
  13. 2026LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimizationcited by 0
  14. 2026AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelinescited by 0
  15. 2026Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?cited by 0
  16. 2026DLoop: Looped Speculative Decodingcited by 0
  17. 2026APEX: Speculate smarter, not deepercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
  2. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  3. 2025Speculative Decoding: Performance or Illusion?
  4. 2026Speculative Decoding with a Speculative Vocabulary
  5. 2024Decoding Speculative Decoding
  6. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  7. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  8. 2024A Theoretical Perspective for Speculative Decoding Algorithm