mlsanityData to 7 Oct 2026

MineDraft: A Framework for Batch Parallel Speculative Decoding

arXiv 2603.18016 ↗First posted 24 Feb 2026cs.CL cs.AI cs.DCcited by 1

On the map: Efficient LLM inference and serving → Speculative decoding

Abstract (arXiv)

Speculative decoding (SD) accelerates large language model inference by using a smaller draft model to propose draft tokens that are subsequently verified by a larger target model. However, the performance of standard SD is often limited by the strictly sequential execution of these drafting and verification stages. To address this, this paper proposes MineDraft, a batch parallel speculative decoding (PSD) framework designed to effectively hide drafting latency by overlapping it with verification. Our theoretical analysis shows that PSD is substantially more efficient than standard SD. MineDraft realizes the PSD through a novel batch-parallel design that maintains two batches of requests, overlapping drafting for one batch with verification for the other. Our experimental results show significant improvements of alg in both throughput (up to 75%) and end-to-end latency (up to 39%) over standard SD. Furthermore, we have implemented MineDraft as a plugin for vLLM, demonstrating its practicality for production-ready inference systems.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Speculative decoding, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 36 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

27 of 36 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms21 May
  2. 2025AdaSpec: Adaptive Speculative Decoding for Fast, SLO-Aware Large Language Model Serving7 Mar
  3. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test3 Mar
  4. 2025TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding21 Feb
  5. 2024Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding20 Nov
  6. 2024SuffixDecoding: Extreme Speculative Decoding for Emerging AI Applications7 Nov
  7. 2024AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability24 Oct
  8. 2024ParallelSpec: Parallel Drafter for Efficient Speculative Decoding8 Oct
  9. 2024MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding20 Aug
  10. 2024PEARL: Parallel Speculative Decoding with Adaptive Draft Length13 Aug
  11. 2024OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure25 Jun
  12. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees24 Jun
7 more
  1. 2024TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput20 Jun
  2. 2024Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference23 May
  3. 2024Recurrent Drafter for Fast Speculative Decoding in Large Language Models14 Mar
  4. 2024Minions: Accelerating Large Language Model Inference with Aggregated Speculative Execution24 Feb
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads19 Jan
  6. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding15 Jan
  7. 2022Fast Inference from Transformers via Speculative Decoding30 Nov

Other references · 8

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
  7. 2023SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefillscited by 187
  8. 2024Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Trainingcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Speculative Decoding: Performance or Illusion?
  2. 2026SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
  3. 2025Speculative Verification: Exploiting Information Gain to Refine Speculative Decoding
  4. 2024Decoding Speculative Decoding
  5. 2026ECHO: Elastic Speculative Decoding with Sparse Gating for High-Concurrency Scenarios
  6. 2025CARD: A Cache-Assisted Parallel Speculative Decoding Framework via Query-and-Correct Paradigm for Accelerating LLM Inference
  7. 2026SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding
  8. 2026DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation