# Accelerated Test-Time Scaling with Model-Free Speculative Sampling

arXiv: https://arxiv.org/abs/2506.04708 · HTML: https://mlsanity.com/paper/2506.04708/

## Key facts

- Cited by 3 papers in the map (data through 7 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)
- Efficient LLM inference and serving / [Speculative decoding](https://mlsanity.com/inference/speculative-decoding/)

## Builds on (most cited first, 26 of 26)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Solving math word problems with process- and outcome-based feedback](https://mlsanity.com/paper/2211.14275/) (2022)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding](https://mlsanity.com/paper/2401.07851/) (2024)
- [Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning](https://mlsanity.com/paper/2405.00451/) (2024)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuning](https://mlsanity.com/paper/2503.07572/) (2025)
- [Reward-Guided Speculative Decoding for Efficient LLM Reasoning](https://mlsanity.com/paper/2501.19324/) (2025)
- [Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning](https://mlsanity.com/paper/2401.10480/) (2024)
- [SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning](https://mlsanity.com/paper/2504.07891/) (2025)
- [Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling](https://mlsanity.com/paper/2408.08696/) (2024)
- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [SAM Decoding: Speculative Decoding via Suffix Automaton](https://mlsanity.com/paper/2411.10666/) (2024)
- [Speculative Decoding and Beyond: An In-Depth Survey of Techniques](https://mlsanity.com/paper/2502.19732/) (2025)
- [Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding](https://mlsanity.com/paper/2404.08698/) (2024)
- [PLD+: Accelerating LLM inference by leveraging Language Model Artifacts](https://mlsanity.com/paper/2412.01447/) (2024)
- [Speculative Decoding for Multi-Sample Inference](https://mlsanity.com/paper/2503.05330/) (2025)
- [POSS: Position Specialist Generates Better Draft for Speculative Decoding](https://mlsanity.com/paper/2506.03566/) (2025)

## Built on by (most cited first, 3 of 3)

- [AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decoding](https://mlsanity.com/paper/2606.05742/) (2026)
- [Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing](https://mlsanity.com/paper/2603.17942/) (2026)
- [Trajectory-Retrieval Speculative Decoding: When Does a Model's Own History Help?](https://mlsanity.com/paper/2610.07350/) (2026)

## Abstract (arXiv)

Language models have demonstrated remarkable capabilities in reasoning tasks through test-time scaling techniques like best-of-N sampling and tree search. However, these approaches often demand substantial computational resources, creating a critical trade-off between performance and efficiency. We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach that exploits the inherent redundancy in reasoning trajectories to achieve significant acceleration without compromising accuracy. Our analysis shows that reasoning paths frequently reuse similar reasoning patterns, enabling efficient model-free token prediction without requiring separate draft models. By introducing stochastic drafting and preserving probabilistic information through a memory-efficient logit-based N-gram module, combined with optimized Gumbel-Top-K sampling and data-driven tree construction, STAND significantly improves token acceptance rates. Extensive evaluations across multiple models and reasoning tasks (AIME-2024, GPQA-Diamond, and LiveCodeBench) demonstrate that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining accuracy. Furthermore, STAND consistently outperforms state-of-the-art speculative decoding methods across diverse inference patterns, including single-trajectory decoding, batch decoding, and test-time tree search. As a model-free approach, STAND can be applied to any existing language model without additional training, making it a powerful plug-and-play solution for accelerating language model reasoning.