mlsanityData to 7 Oct 2026

Speculative decoding

522 papers9 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: decoding, speculative, speculative decoding, draft, target

Key facts. Speculative decoding: 522 papers on arXiv since 2021 (data through 7 Oct 2026). 465 of its 522 mapped papers (89%) cite “Fast Inference from Transformers via Speculative Decoding” (2022, arXiv 2211.17192). 382 of its 522 mapped papers (73%) cite “Accelerating Large Language Model Decoding with Speculative Sampling” (2023, arXiv 2302.01318). 28 new papers 24 Sep–7 Oct.

What’s new

24 Sep – 7 Oct 2026

28 new papers, 24 Sep – 7 Oct; 9 this week.

  1. This week · 9
  2. Speedbumps: Rejection Attacks on Speculative Decoding
  3. Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Rounds
  4. Secure Speculative Decoding for Large Language Models
  5. Nucleus Speculative Decoding: Plausibility-Aware Verification Beyond Exact Distribution
  6. APEX: Speculate smarter, not deeper
  7. DLoop: Looped Speculative Decoding
  8. Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?
  9. SchemaFill: Efficient LLM Tool Calling via Slot-Parallel Speculative Decoding also fits: Serving systems and request scheduling
  10. D-Loop: Looped Diffusion Drafting for Speculative Decoding
  11. Earlier · 19
  12. SharpDraft: Accelerating Long-Context Speculative Decoding with Cardinality-Aware Query Scaling
All 28
  1. BitNest: Bit-Nested Speculative Decoding for Memory-Efficient LLM Inference Acceleration
  2. DRelay: Global Draft Context for Prefix-Aware Parallel Speculative Decoding Repair
  3. AgSpec: Pushing the Limits of Retrieval-Based Speculative Decoding in Coding Agent Pipelines
  4. HAWK: Rethinking Multimodal Drafting for Speculative Decoding also fits: Fast autoregressive visual generation
  5. UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decoding
  6. Recovering Off-Policy Supervision for Speculative Decoding
  7. CAST: Cost-Aware Speculative Trees from One-Pass Block Drafters
  8. DEdit: Iterative Draft Editing for Speculative Decoding
  9. AS2D: Accelerating On-Demand Audio Understanding on Mobile Devices
  10. DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification
  11. LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
  12. SEED: Self-Speculative Decoding via Implicit Encoder-Decoder
  13. Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding
  14. BV Loss: Block Verification-Aware Loss for Block Diffusion Speculative Decoding
  15. Reciprocal Guidance: Orchestrating Draft and Verify Budgets for Advancing the Diffusion-AR Self-Speculation Frontier
  16. Tsubame: Tree Replay for Diffusion-Based Speculative Decoding
  17. Acoustic Progress Propagation for Long-Horizon Speculative Decoding in ASR
  18. Mentored Decoding: Faster Inference meets Boosting

Foundations

Most cited by papers here

  1. Fast Inference from Transformers via Speculative Decoding2022cited by 465 here
  2. Accelerating Large Language Model Decoding with Speculative Sampling2023cited by 382 here
  3. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads2024cited by 353 here
  4. EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty2024cited by 296 here
  5. EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees2024cited by 224 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022Fast Inference from Transformers via Speculative Decodingcited by 465 here
  2. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 382 here
  3. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 87 here
  4. 2023Online Speculative Decodingcited by 68 here
  5. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 109 here
  6. 2023Cascade Speculative Drafting for Even Faster LLM Inferencecited by 61 here
  7. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 296 here
  8. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 167 here
  9. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 73 here
  10. 2024TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decodingcited by 61 here
  11. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 224 here
  12. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 182 here
  13. 2026DFlash: Block Diffusion for Flash Speculative Decodingcited by 58 here
11 more it builds on, not in this chain
  1. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 94 here
  2. 2023Speculative Decoding with Big Little Decodercited by 84 here
  3. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 210 here
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 160 here
  5. 2023SpecTr: Fast Speculative Decoding via Optimal Transportcited by 84 here
  6. 2023REST: Retrieval-Based Speculative Decodingcited by 112 here
  7. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 163 here
  8. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 353 here
  9. 2024Sequoia: Scalable, Robust, and Hardware-aware Speculative Decodingcited by 67 here
  10. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 88 here
  11. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Lengthcited by 63 here

The 24 most cited of 546.

Browse

30 of 522, ranked 6–35

  1. 6SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification2023210
  2. 7EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test2025182
  3. 8Break the Sequential Dependency of LLM Inference Using Lookahead Decoding2024167
  4. 9Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding2024163
  5. 10REST: Retrieval-Based Speculative Decoding2023112
  6. 11DistillSpec: Improving Speculative Decoding via Knowledge Distillation2023109
  7. 12Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation202294
  8. 13Accelerating LLM Inference with Staged Speculative Decoding202387
  9. 14Speculative Decoding with Big Little Decoder202384
  10. 15SpecTr: Fast Speculative Decoding via Optimal Transport202384
All 30
  1. 16Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding202473
  2. 17Online Speculative Decoding202368
  3. 18Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding202467
  4. 19PEARL: Parallel Speculative Decoding with Adaptive Draft Length202463
  5. 20TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding202461
  6. 21Cascade Speculative Drafting for Even Faster LLM Inference202361
  7. 22DFlash: Block Diffusion for Flash Speculative Decoding202658
  8. 23MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding202457
  9. 24GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding202456
  10. 25Accelerating Transformer Inference for Translation via Parallel Decoding202355
  11. 26SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths202454
  12. 27OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure202453
  13. 28Learning Harmonized Representations for Speculative Sampling202450
  14. 29Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment202545
  15. 30SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration202444
  16. 31Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting202434
  17. 32SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices202431
  18. 33Multi-Candidate Speculative Decoding202427
  19. 34Block Verification Accelerates Speculative Decoding202426
  20. 35SAM Decoding: Speculative Decoding via Suffix Automaton202426