mlsanityData to 14 Oct 2026

SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning

arXiv 2504.07891 ↗First posted 10 Apr 2025cs.LG cs.AIcited by 59

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs). However, this improved accuracy comes at the cost of high inference latency due to the length of generated reasoning sequences and the autoregressive nature of decoding. Our key insight in tackling these overheads is that LRM inference, and the reasoning that it embeds, is highly tolerant of approximations: complex tasks are typically broken down into simpler steps, each of which brings utility based on the semantic insight it provides for downstream steps rather than the exact tokens it generates. Accordingly, we introduce SpecReason, a system that automatically accelerates LRM inference by using a lightweight model to (speculatively) carry out simpler intermediate reasoning steps and reserving the costly base model only to assess (and potentially correct) the speculated outputs. Importantly, SpecReason's focus on exploiting the semantic flexibility of thinking tokens in preserving final-answer accuracy is complementary to prior speculation techniques, most notably speculative decoding, which demands token-level equivalence at each step. Across a variety of reasoning benchmarks, SpecReason achieves 1.4-3.0× speedup over vanilla LRM inference while improving accuracy by 0.4-9.0%. Compared to speculative decoding without SpecReason, their combination yields an additional 8.8-58.0% latency reduction. We open-source SpecReason at https://github.com/ruipeterpan/specreason.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 24 of its 37 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

24 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2023Let's Verify Step by Stepcited by 3,789
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  12. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
11 more
  1. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  2. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  4. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  5. 2023REST: Retrieval-Based Speculative Decodingcited by 150
  6. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solverscited by 138
  7. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  8. 2024Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recyclingcited by 47
  9. 2023Lookahead: An Inference Acceleration Framework for Large Language Model with Lossless Generation Accuracycited by 37
  10. 2024Decoding Speculative Decodingcited by 34
  11. 2024Marconi: Prefix Caching for the Era of Hybrid LLMscited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 28; 2026: 31.

59 papers.

Sort:

By area:

  1. 2025Efficient Reasoning Models: A Surveycited by 95
  2. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 48
  3. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  4. 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routingcited by 29
  5. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 27
  6. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficientcited by 24
  7. 2026GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughtscited by 17
  8. 2025TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scalingcited by 14
  9. 2025Taming the Long-Tail: Efficient Reasoning RL Training with Adaptive Draftercited by 14
  10. 2025AutoJudge: Judge Decoding Without Manual Annotationcited by 13
  11. 2025Scaling Speculative Decoding with Lookahead Reasoningcited by 13
  12. 2025Done Is Better than Perfect: Unlocking Efficient Reasoning by Structured Multi-Turn Decompositioncited by 11
38 more
  1. 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
  2. 2025Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoningcited by 9
  3. 2025Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculationcited by 8
  4. 2026RelayLLM: Efficient Reasoning via Collaborative Decodingcited by 8
  5. 2026TRIM: Hybrid Inference via Targeted Stepwise Routing in Multi-Step Reasoning Taskscited by 8
  6. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 8
  7. 2026DualSpec: Accelerating Deep Research Agents via Dual-Process Action Speculationcited by 8
  8. 2025Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scalingcited by 6
  9. 2026Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agentscited by 6
  10. 2025Fast Quiet-STaR: Thinking Without Thought Tokenscited by 5
  11. 2026InfoSeeker: A Scalable Hierarchical Parallel Agent Framework for Web Information Seekingcited by 5
  12. 2025SSR: Speculative Parallel Scaling Reasoning in Test-timecited by 4
  13. 2025Efficient Tree-Structured Deep Research with Adaptive Resource Allocationcited by 4
  14. 2025Reject Only Critical Tokens: Pivot-Aware Speculative Decodingcited by 4
  15. 2025Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMscited by 4
  16. 2026ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verificationcited by 4
  17. 2025FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoningcited by 3
  18. 2025ATTS: Asynchronous Test-Time Scaling via Conformal Predictioncited by 3
  19. 2025MixReasoning: Switching Modes to Thinkcited by 3
  20. 2025Entropy-Aware Token Rejection for Improving Speculative Decodingcited by 3
  21. 2026Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal Statescited by 3
  22. 2026Select to Think: Unlocking SLM Potential with Local Sufficiencycited by 3
  23. 2026Speculate with Memory: Lossless Acceleration for LLM Agentscited by 3
  24. 2025POSS: Position Specialist Generates Better Draft for Speculative Decodingcited by 2
  25. 2025Accelerated Test-Time Scaling with Model-Free Speculative Samplingcited by 2
  26. 2025Thinking with Drafts: Speculative Temporal Reasoning for Efficient Long Video Understandingcited by 2
  27. 2025Arbitrage: Efficient Reasoning via Advantage-Aware Speculationcited by 2
  28. 2026Rethinking Model Efficiency: Multi-Agent Inference with Large Modelscited by 2
  29. 2026Rubric-Guided Process Reward for Stepwise Model Routingcited by 2
  30. 2026SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planningcited by 1
  31. 2026TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Modelscited by 1
  32. 2026RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoningcited by 1
  33. 2026UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scalingcited by 1
  34. 2026Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recoverycited by 1
  35. 2026FutureBridge: Token Selection Beyond Local Preference in Collaborative Decodingcited by 1
  36. 2026Beyond Model Scaling: Test-Time Intervention for Efficient Deep Reasoningcited by 0
  37. 2026Precedent-Informed Reasoning: Mitigating Overthinking in Large Reasoning Models via Test-Time Precedent Learningcited by 0
  38. 2026From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoningcited by 0

50 most cited of 59 All 59 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SpecExit: Accelerating Large Reasoning Model via Speculative Exit
  2. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time
  3. 2025ThinkBrake: Efficient Reasoning via Log-Probability Margin Guided Decoding
  4. 2024SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding
  5. 2025Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
  6. 2025A*-Decoding: Token-Efficient Inference Scaling
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  8. 2026Taming Speculative Search for Test-Time Scaling in LLM Serving