# AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures

arXiv: https://arxiv.org/abs/2412.18910 · HTML: https://mlsanity.com/paper/2412.18910/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving.
- Builds on 16 of its 35 references that are in the map.
- First posted on arXiv 25 Dec 2024.

## Placed in

- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 16 of 16)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [A Survey on Efficient Inference for Large Language Models](https://mlsanity.com/paper/2404.14294/) (2024)
- [Accelerating LLM Inference with Staged Speculative Decoding](https://mlsanity.com/paper/2308.04623/) (2023)
- [Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding](https://mlsanity.com/paper/2402.12374/) (2024)
- [PEARL: Parallel Speculative Decoding with Adaptive Draft Length](https://mlsanity.com/paper/2408.11850/) (2024)
- [SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths](https://mlsanity.com/paper/2405.19715/) (2024)
- [OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure](https://mlsanity.com/paper/2406.17276/) (2024)
- [Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting](https://mlsanity.com/paper/2404.18911/) (2024)
- [Do language models plan ahead for future tokens?](https://mlsanity.com/paper/2404.00859/) (2024)
- [Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models](https://mlsanity.com/paper/2405.04304/) (2024)
- [Dynamic Depth Decoding: Faster Speculative Decoding for LLMs](https://mlsanity.com/paper/2409.00142/) (2024)

## Built on by (most cited first, 19 of 19)

- [FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling](https://mlsanity.com/paper/2502.14856/) (2025)
- [LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation](https://mlsanity.com/paper/2507.01449/) (2025)
- [TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees](https://mlsanity.com/paper/2601.07353/) (2026)
- [C2T: A Classifier-Based Tree Construction Method in Speculative Decoding](https://mlsanity.com/paper/2502.13652/) (2025)
- [Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Serving](https://mlsanity.com/paper/2512.22420/) (2025)
- [Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding](https://mlsanity.com/paper/2605.20104/) (2026)
- [Learning to Draft: Adaptive Speculative Decoding with Reinforcement Learning](https://mlsanity.com/paper/2603.01639/) (2026)
- [D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding](https://mlsanity.com/paper/2607.14647/) (2026)
- [AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters](https://mlsanity.com/paper/2607.19223/) (2026)
- [HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding](https://mlsanity.com/paper/2505.13254/) (2025)
- [Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting](https://mlsanity.com/paper/2605.29727/) (2026)
- [TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts](https://mlsanity.com/paper/2608.22788/) (2026)
- [Polybasic Speculative Decoding Through a Theoretical Perspective](https://mlsanity.com/paper/2510.26527/) (2025)
- [TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verification](https://mlsanity.com/paper/2601.23180/) (2026)
- [CURE: Local Uncertainty Repair for Block-Parallel Speculative Decoding](https://mlsanity.com/paper/2608.00531/) (2026)
- [Margins, Not Windows: Training-Free Per-Step Lossy Speculative Decoding](https://mlsanity.com/paper/2609.02897/) (2026)
- [The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts](https://mlsanity.com/paper/2609.22156/) (2026)
- [DLoop: Looped Speculative Decoding](https://mlsanity.com/paper/2610.07659/) (2026)
- [APEX: Speculate smarter, not deeper](https://mlsanity.com/paper/2610.07780/) (2026)

## Abstract (arXiv)

Speculative Decoding (SD) is a popular lossless technique for accelerating the inference of Large Language Models (LLMs). We show that the decoding speed of SD frameworks with static draft structures can be significantly improved by incorporating context-aware adaptive draft structures. However, current studies on adaptive draft structures are limited by their performance, modeling approaches, and applicability. In this paper, we introduce AdaEAGLE, the first SD framework that explicitly models adaptive draft structures. AdaEAGLE leverages the Lightweight Draft Length Predictor (LDLP) module to explicitly predict the optimal number of draft tokens during inference to guide the draft model. It achieves comparable speedup results without manual thresholds and allows for deeper, more specialized optimizations. Moreover, together with threshold-based strategies, AdaEAGLE achieves a 1.62× speedup over the vanilla AR decoding and outperforms fixed-length SotA baseline while maintaining output quality.