mlsanityData to 7 Oct 2026

Tutorial Proposal: Speculative Decoding for Efficient LLM Inference

arXiv 2503.00491 ↗First posted 1 Mar 2025cs.CLcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

This tutorial presents a comprehensive introduction to Speculative Decoding (SD), an advanced technique for LLM inference acceleration that has garnered significant research interest in recent years. SD is introduced as an innovative decoding paradigm to mitigate the high inference latency stemming from autoregressive decoding in LLMs. At each decoding step, SD efficiently drafts several future tokens and then verifies them in parallel. This approach, unlike traditional autoregressive decoding, facilitates the simultaneous decoding of multiple tokens per step, thereby achieving promising 2x-4x speedups in LLM inference while maintaining original distributions. This tutorial delves into the latest techniques in SD, including draft model architectures and verification strategies. Additionally, it explores the acceleration potential and future research directions in this promising field. We aim for this tutorial to elucidate the current research landscape and offer insights for researchers interested in Speculative Decoding, ultimately contributing to more efficient LLM inference.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 32 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

23 of 32 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
  2. 2024GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding3 Feb
  3. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  6. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  7. 2023REST: Retrieval-Based Speculative Decoding14 Nov
  8. 2023Accelerating Transformer Inference for Translation via Parallel Decoding17 May
  9. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  10. 2023Speculative Decoding with Big Little Decoder15 Feb
  11. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  12. 2022Fast Inference from Transformers via Speculative Decoding30 Nov
2 more
  1. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation30 Mar
  2. 2021Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding9 Jun

Other references · 9

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  6. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  7. 2024Speculative RAG: Enhancing Retrieval Augmented Generation through Draftingcited by 76
  8. 2023LLMCad: Fast and Scalable On-device Large Language Model Inferencecited by 45
  9. 2024Accelerating Retrieval-Augmented Language Model Serving with Speculationcited by 20

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Cognitive Load-Aware Inference: A Neuro-Symbolic Framework for Optimizing the Token Economy of Large Language Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
  2. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding
  3. 2025CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference
  4. 2025Speculative Decoding: Performance or Illusion?
  5. 2026Faster LLM Inference via Sequential Monte Carlo
  6. 2025Mirror Speculative Decoding: Breaking the Serial Barrier in LLM Inference
  7. 2023Accelerating LLM Inference with Staged Speculative Decoding
  8. 2025Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios