mlsanityData to 14 Oct 2026

AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures

arXiv 2412.18910 ↗First posted 25 Dec 2024cs.AI cs.CLcited by 18

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative Decoding (SD) is a popular lossless technique for accelerating the inference of Large Language Models (LLMs). We show that the decoding speed of SD frameworks with static draft structures can be significantly improved by incorporating context-aware adaptive draft structures. However, current studies on adaptive draft structures are limited by their performance, modeling approaches, and applicability. In this paper, we introduce AdaEAGLE, the first SD framework that explicitly models adaptive draft structures. AdaEAGLE leverages the Lightweight Draft Length Predictor (LDLP) module to explicitly predict the optimal number of draft tokens during inference to guide the draft model. It achieves comparable speedup results without manual thresholds and allows for deeper, more specialized optimizations. Moreover, together with threshold-based strategies, AdaEAGLE achieves a 1.62× speedup over the vanilla AR decoding and outperforms fixed-length SotA baseline while maintaining output quality.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 35 references that are in the map. First posted on arXiv 25 Dec 2024.

Builds on

16 of 35 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Dynamic Depth Decoding: Faster Speculative Decoding for LLMs30 Aug
  2. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  3. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure25 Jun
  4. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  5. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths30 May
  6. 2024Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models7 May
  7. 2024Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting29 Apr
  8. 2024Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding19 Feb
  9. 2023Accelerating LLM Inference with Staged Speculative Decoding8 Aug
  10. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb

Other references · 6

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  6. 2024Do language models plan ahead for future tokens?cited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 12.

18 papers.

Sort:

By area:

  1. 2025FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Samplingcited by 20
  2. 2025LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculationcited by 14
  3. 2026TALON: Confidence-Aware Speculative Decoding with Adaptive Token Treescited by 13
  4. 2025C2T: A Classifier-Based Tree Construction Method in Speculative Decodingcited by 9
  5. 2025Nightjar: Dynamic Adaptive Speculative Decoding for Large Language Models Servingcited by 6
  6. 2026Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decodingcited by 6
  7. 2026D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decodingcited by 5
  8. 2026AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafterscited by 4
  9. 2025HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decodingcited by 2
  10. 2026Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Draftingcited by 2
  11. 2026TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rolloutscited by 2
  12. 2025Polybasic Speculative Decoding Through a Theoretical Perspectivecited by 1
6 more
  1. 2026TriSpec: Ternary Speculative Decoding via Lightweight Proxy Verificationcited by 1
  2. 2026CURE: Local Uncertainty Repair for Block-Parallel Speculative Decodingcited by 1
  3. 2026Margins, Not Windows: Training-Free Per-Step Lossy Speculative Decodingcited by 1
  4. 2026The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Expertscited by 1
  5. 2026DLoop: Looped Speculative Decodingcited by 0
  6. 2026APEX: Speculate smarter, not deepercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  2. 2024AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability
  3. 2024Decoding Speculative Decoding
  4. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs
  5. 2024Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity
  6. 2025Speculative Decoding: Performance or Illusion?
  7. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  8. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding