mlsanityData to 7 Oct 2026

Multi-Drafter Speculative Decoding with Alignment Feedback

arXiv 2604.05417 ↗First posted 7 Apr 2026cs.CLcited by 3

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding (SD) accelerates large language model (LLM) inference by using a smaller model to draft future tokens, which are then verified by the target LLM. This preserves generation quality by accepting only aligned tokens. However, individual drafters, often trained for specific tasks or domains, exhibit limited effectiveness across diverse applications. To address this, we introduce textscMetaSD, a unified framework that integrates multiple drafters into the SD process. MetaSD dynamically allocates computational resources to heterogeneous drafters by leveraging alignment feedback and framing drafter selection as a multi-armed bandit problem. Extensive experiments show MetaSD consistently outperforms single-drafter approaches.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 38 of its 83 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

38 of 83 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms21 May
  2. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoning31 Jan
  3. 2024A Theoretical Perspective for Speculative Decoding Algorithm30 Oct
  4. 2024Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters24 Jun
  5. 2024Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism6 Jun
  6. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decoding3 Feb
  7. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  8. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  9. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  10. 2024Multi-Candidate Speculative Decoding12 Jan
  11. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  12. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  8. 2015Teaching Machines to Read and Comprehendcited by 1,039
  9. 2016Sequence-Level Knowledge Distillationcited by 761
  10. 2020Aligning AI With Shared Human Valuescited by 706
  11. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  12. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
14 more
  1. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 318
  2. 2016Optimal Best Arm Identification with Fixed Confidencecited by 233
  3. 2013lil' UCB : An Optimal Exploration Algorithm for Multi-Armed Banditscited by 148
  4. 2019Batched Multi-armed Bandits Problemcited by 97
  5. 2016Tight (Lower) Bounds for the Fixed Budget Best Arm Identification Bandit Problemcited by 79
  6. 2019Regret Bounds for Batched Banditscited by 49
  7. 2013Bandits with Switching Costs: T^{2/3} Regretcited by 38
  8. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 35
  9. 2016Towards Instance Optimal Bounds for Best Arm Identificationcited by 30
  10. 2024Which LLM to Play? Convergence-Aware Online Model Selection with Time-Increasing Banditscited by 26
  11. 2022Revisiting Simple Regret: Fast Rates for Returning a Good Armcited by 25
  12. 2021An Algorithm for Stochastic and Adversarial Bandits with Switching Costscited by 18
  13. 2022JParaCrawl v3.0: A Large-scale English-Japanese Parallel Corpuscited by 17
  14. 2023Budgeted Multi-Armed Bandits with Asymmetric Confidence Intervalscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routingcited by 1
  2. 2026AS2D: Accelerating On-Demand Audio Understanding on Mobile Devicescited by 0
  3. 2026CAST: Cost-Aware Speculative Trees from One-Pass Block Drafterscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Steering Pretrained Drafters during Speculative Decoding
  2. 2024SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding
  3. 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
  4. 2025Mamba Drafters for Speculative Decoding
  5. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment
  6. 2025Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification
  7. 2025SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding
  8. 2026D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models