# Dynamic Depth Decoding: Faster Speculative Decoding for LLMs

arXiv: https://arxiv.org/abs/2409.00142 · HTML: https://mlsanity.com/paper/2409.00142/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [PEARL: Parallel Speculative Decoding with Adaptive Draft Length](https://mlsanity.com/paper/2408.11850/) (2024)
- [SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths](https://mlsanity.com/paper/2405.19715/) (2024)
- [TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput](https://mlsanity.com/paper/2406.14066/) (2024)

## Built on by (most cited first, 20 of 20)

- [A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models](https://mlsanity.com/paper/2508.08712/) (2025)
- [DySpec: Faster Speculative Decoding with Dynamic Token Tree Structure](https://mlsanity.com/paper/2410.11744/) (2024)
- [Speculative Decoding and Beyond: An In-Depth Survey of Techniques](https://mlsanity.com/paper/2502.19732/) (2025)
- [AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures](https://mlsanity.com/paper/2412.18910/) (2024)
- [Grouped Speculative Decoding for Autoregressive Image Generation](https://mlsanity.com/paper/2508.07747/) (2025)
- [Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation](https://mlsanity.com/paper/2411.18462/) (2024)
- [ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios](https://mlsanity.com/paper/2604.09603/) (2026)
- [MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts](https://mlsanity.com/paper/2511.14102/) (2025)
- [C2T: A Classifier-Based Tree Construction Method in Speculative Decoding](https://mlsanity.com/paper/2502.13652/) (2025)
- [Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding](https://mlsanity.com/paper/2503.10135/) (2025)
- [Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving](https://mlsanity.com/paper/2512.22420/) (2025)
- [Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding](https://mlsanity.com/paper/2605.20104/) (2026)
- [Speculative Coupled Decoding for Training-Free Lossless Acceleration of Autoregressive Visual Generation](https://mlsanity.com/paper/2510.24211/) (2025)
- [Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding](https://mlsanity.com/paper/2605.00342/) (2026)
- [HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding](https://mlsanity.com/paper/2505.13254/) (2025)
- [PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length](https://mlsanity.com/paper/2602.01274/) (2026)
- [TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts](https://mlsanity.com/paper/2608.22788/) (2026)
- [Dynamic Delayed Tree Expansion For Improved Multi-Path Speculative Decoding](https://mlsanity.com/paper/2602.16994/) (2026)
- [Parallel Prefix Verification for Speculative Generation](https://mlsanity.com/paper/2605.04263/) (2026)
- [DLoop: Looped Speculative Decoding](https://mlsanity.com/paper/2610.07659/) (2026)

## Abstract (arXiv)

The acceleration of Large Language Models (LLMs) with speculative decoding provides a significant runtime improvement without any loss of accuracy. Currently, EAGLE-2 is the state-of-the-art speculative decoding method, improving on EAGLE with a dynamic draft tree. We introduce Dynamic Depth Decoding (DDD), which optimises EAGLE-2's tree drafting method using a dynamic depth. This extends the average speedup that EAGLE-2 achieves over EAGLE by 44%, giving DDD an average speedup of 3.16x.