mlsanityData to 14 Oct 2026

Dynamic Depth Decoding: Faster Speculative Decoding for LLMs

arXiv 2409.00142 ↗First posted 30 Aug 2024cs.CL cs.AIcited by 20

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

The acceleration of Large Language Models (LLMs) with speculative decoding provides a significant runtime improvement without any loss of accuracy. Currently, EAGLE-2 is the state-of-the-art speculative decoding method, improving on EAGLE with a dynamic draft tree. We introduce Dynamic Depth Decoding (DDD), which optimises EAGLE-2's tree drafting method using a dynamic depth. This extends the average speedup that EAGLE-2 achieves over EAGLE by 44%, giving DDD an average speedup of 3.16x.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 7 of its 9 references that are in the map. First posted on arXiv 30 Aug 2024.

Builds on

7 of 9 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  2. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  3. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput20 Jun
  4. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths30 May
  5. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  6. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 1

  1. 2020Language Models are Few-Shot Learnerscited by 32,419

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 8.

20 papers.

Sort:

By area:

  1. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  2. 2024DySpec: Faster Speculative Decoding with Dynamic Token Tree Structurecited by 25
  3. 2025Speculative Decoding and Beyond: An In-Depth Survey of Techniquescited by 23
  4. 2024AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structurescited by 18
  5. 2025Grouped Speculative Decoding for Autoregressive Image Generationcited by 17
  6. 2024Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generationcited by 16
  7. 2026ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarioscited by 12
  8. 2025MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Expertscited by 11
  9. 2025C2T: A Classifier-Based Tree Construction Method in Speculative Decodingcited by 9
  10. 2025Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decodingcited by 9
  11. 2025Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Servingcited by 6
  12. 2026Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decodingcited by 6
8 more
  1. 2025Speculative Coupled Decoding for Training-Free Lossless Acceleration of Autoregressive Visual Generationcited by 4
  2. 2026Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decodingcited by 4
  3. 2025HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decodingcited by 2
  4. 2026PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Lengthcited by 2
  5. 2026TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rolloutscited by 2
  6. 2026Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decodingcited by 0
  7. 2026Parallel Prefix Verification for Speculative Generationcited by 0
  8. 2026DLoop: Looped Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Inference-Cost-Aware Dynamic Tree Construction for Efficient Inference in Large Language Models
  2. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  3. 2024Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement
  4. 2025MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE
  5. 2023Accelerating LLM Inference with Staged Speculative Decoding
  6. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  7. 2024Decoding Speculative Decoding
  8. 2025Speculative Decoding: Performance or Illusion?