mlsanityData to 7 Oct 2026

Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding

arXiv 2411.13157 ↗First posted 20 Nov 2024cs.CL cs.AI cs.LGcited by 7

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Efficient inference in large language models (LLMs) has become a critical focus as their scale and complexity grow. Traditional autoregressive decoding, while effective, suffers from computational inefficiencies due to its sequential token generation process. Speculative decoding addresses this bottleneck by introducing a two-stage framework: drafting and verification. A smaller, efficient model generates a preliminary draft, which is then refined by a larger, more sophisticated model. This paper provides a comprehensive survey of speculative decoding methods, categorizing them into draft-centric and model-centric approaches. We discuss key ideas associated with each method, highlighting their potential for scaling LLM inference. This survey aims to guide future research in optimizing speculative decoding and its integration into real-world LLM applications.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 28 of its 34 references that are in the map. First posted on arXiv 20 Nov 2024.

Builds on

28 of 34 references.

Closest ancestors · 19

Same direction, newest first

  1. 2024Learning Harmonized Representations for Speculative Sampling28 Aug
  2. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding20 Aug
  3. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure25 Jun
  4. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  5. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput20 Jun
  6. 2024SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices4 Jun
  7. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths30 May
  8. 2024Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference28 May
  9. 2024BASS: Batched Attention-optimized Speculative Sampling24 Apr
  10. 2024Exploring and Improving Drafts in Blockwise Parallel Decoding14 Apr
  11. 2024Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs29 Feb
  12. 2024Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens24 Feb
7 more
  1. 2024Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement21 Feb
  2. 2024ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding21 Feb
  3. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding7 Feb
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  6. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  7. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  4. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  5. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  6. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  7. 2024A Thorough Examination of Decoding Methods in the Era of LLMscited by 112
  8. 2023Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracycited by 37
  9. 2024Aladdin: Joint Placement and Scaling for SLO-Aware LLM Servingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  2. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  3. 2025TETRIS: Optimal Draft Token Selection for Batch Speculative Decodingcited by 14
  4. 2025DReSD: Dense Retrieval for Speculative Decodingcited by 6
  5. 2025Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)cited by 2
  6. 2026MineDraft: A Framework for Batch Parallel Speculative Decodingcited by 1
  7. 2026SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decodingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024A Theoretical Perspective for Speculative Decoding Algorithm
  2. 2025Speculative Decoding: Performance or Illusion?
  3. 2025Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
  4. 2025Tutorial Proposal: Speculative Decoding for Efficient LLM Inference
  5. 2026Faster LLM Inference via Sequential Monte Carlo
  6. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  7. 2023Accelerating LLM Inference with Staged Speculative Decoding
  8. 2024Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models