mlsanityData to 14 Oct 2026

Speculative Decoding for Multi-Sample Inference

arXiv 2503.05330 ↗First posted 7 Mar 2025cs.CL cs.AIcited by 5

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

We propose a novel speculative decoding method tailored for multi-sample reasoning scenarios, such as self-consistency and Best-of-N sampling. Our method exploits the intrinsic consensus of parallel generation paths to synthesize high-quality draft tokens without requiring auxiliary models or external databases. By dynamically analyzing structural patterns across parallel reasoning paths through a probabilistic aggregation mechanism, it identifies consensus token sequences that align with the decoding distribution. Evaluations on mathematical reasoning benchmarks demonstrate a substantial improvement in draft acceptance rates over baselines, while reducing the latency in draft token construction. This work establishes a paradigm shift for efficient multi-sample inference, enabling seamless integration of speculative decoding with sampling-based reasoning techniques.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 25 of its 34 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

25 of 34 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation27 Feb
  2. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
  3. 2024Improving Self Consistency in LLMs through Probabilistic Tokenization4 Jul
  4. 2024Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation2 Jul
  5. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  6. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  7. 2023Universal Self-Consistency for Large Language Model Generation29 Nov
  8. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
  9. 2023Self-Evaluation Guided Beam Search for Reasoning1 May

Other references · 16

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  8. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  9. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  10. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  11. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  12. 2022Making Large Language Models Better Reasoners with Step-Aware Verifiercited by 233
4 more
  1. 2023REST: Retrieval-Based Speculative Decodingcited by 150
  2. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  3. 2023Turning Dust into Gold: Distilling Complex Reasoning Capabilities from LLMs by Leveraging Negative Datacited by 21
  4. 2024BASS: Batched Attention-optimized Speculative Samplingcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2026ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decodingcited by 3
  2. 2025Accelerated Test-Time Scaling with Model-Free Speculative Samplingcited by 2
  3. 2025Hierarchical Verification of Speculative Beams for Accelerating LLM Inferencecited by 1
  4. 2026UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inferencecited by 1
  5. 2026Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Schedulingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026TAPS: Task Aware Proposal Distributions for Speculative Sampling
  2. 2026From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning
  3. 2024SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding
  4. 2026Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
  5. 2025Scaling Speculative Decoding with Lookahead Reasoning
  6. 2026Multi-Drafter Speculative Decoding with Alignment Feedback
  7. 2026LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
  8. 2024Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation