mlsanityData to 14 Oct 2026

DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification

arXiv 2604.07622 ↗First posted 8 Apr 2026cs.CL cs.AI cs.LGcited by 4

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding is an effective technique for accelerating large language model inference by drafting multiple tokens in parallel. In practice, its speedup is often bottlenecked by a rigid verification step that strictly enforces the accepted token distribution to exactly match the target model. This constraint leads to the rejection of many plausible tokens, lowering the acceptance rate and limiting overall time speedup. To overcome this limitation, we propose Dynamic Verification Relaxed Speculative Decoding (DIVERSED), a relaxed verification framework that improves time efficiency while preserving generation quality. DIVERSED learns an ensemble-based verifier that blends the draft and target model distributions with a task-dependent and context-dependent weight. We provide theoretical justification for our approach and demonstrate empirically that DIVERSED achieves substantially higher inference efficiency compared to standard speculative decoding methods. Code is available at: https://github.com/comeusr/diversed.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 34 of its 44 references that are in the map. First posted on arXiv 8 Apr 2026.

Builds on

34 of 44 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment31 Jan
  2. 2024A Theoretical Perspective for Speculative Decoding Algorithm30 Oct
  3. 2024SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths30 May
  4. 2024Nearest Neighbor Speculative Decoding for LLM Generation and Attribution29 May
  5. 2024Faster Cascades via Speculative Decoding29 May
  6. 2024EMS-SD: Efficient Multi-sample Speculative Decoding for Accelerating Large Language Models13 May
  7. 2024Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models7 May
  8. 2024Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs29 Feb
  9. 2024Decoding Speculative Decoding2 Feb
  10. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  12. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillation12 Oct
4 more
  1. 2023Online Speculative Decoding11 Oct
  2. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification16 May
  3. 2023Accelerating Large Language Model Decoding with Speculative Sampling2 Feb
  4. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  11. 2025Gemma 3 Technical Reportcited by 1,824
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
6 more
  1. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,292
  2. 2024A Survey on Large Language Models for Code Generationcited by 658
  3. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 307
  4. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalizationcited by 241
  5. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  6. 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026A Practical Investigation of Training-free Relaxed Speculative Decodingcited by 3
  2. 2026Fast-dLLM++: Fr'echet Profile Decoding for Faster Diffusion LLM Inferencecited by 2
  3. 2026Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modescited by 0
  4. 2026UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization
  2. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  3. 2025AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference
  4. 2025Speeding up Speculative Decoding via Sequential Approximate Verification
  5. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
  6. 2025Speculative Decoding: Performance or Illusion?
  7. 2026PACER: Blockwise Pre-verification for Speculative Decoding with Adaptive Length
  8. 2024Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion