mlsanityData to 7 Oct 2026

Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification

arXiv 2502.01839 ↗First posted 3 Feb 2025cs.LG cs.AIcited by 49

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Sampling-based search, a simple paradigm for utilizing test-time compute, involves generating multiple candidate responses and selecting the best one -- typically by having models self-verify each response for correctness. In this paper, we study the scaling trends governing sampling-based search. Among our findings is that simply scaling up a minimalist implementation of sampling-based search, using only random sampling and direct self-verification, provides a practical inference method that, for example, elevates the reasoning capabilities of Gemini v1.5 Pro above that of o1-Preview on popular benchmarks. We partially attribute the scalability of sampling-based search to a phenomenon of implicit scaling, where sampling a larger pool of responses in turn improves self-verification accuracy. We further identify two useful principles for improving self-verification capabilities with test-time compute: (1) comparing across responses provides helpful signals about the locations of errors and hallucinations, and (2) different model output styles are useful for different contexts -- chains of thought are useful for reasoning but harder to verify. We also find that, though accurate verification can be elicited, frontier models demonstrate remarkably weak out-of-box verification capabilities and introduce a benchmark to measure progress on these deficiencies.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 48 of its 52 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

48 of 52 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models18 Dec
  2. 2024Provable Scaling Laws for the Test-Time Compute of Large Language Models29 Nov
  3. 2024Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoning14 Oct
  4. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  6. 2024Large Language Models have Intrinsic Self-Correction Ability21 Jun
  7. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks12 Feb
  8. 2023LLMs cannot find reasoning errors, but can correct them given the error location14 Nov
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  12. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
3 more
  1. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  2. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  3. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  10. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  11. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  12. 2022Self-critiquing models for assisting human evaluatorscited by 372
21 more
  1. 2023Language Models can Solve Computer Taskscited by 371
  2. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  3. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  4. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  5. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 304
  6. 2022Generating Sequences by Learning to Self-Correctcited by 271
  7. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  8. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  9. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  10. 2024Language Models Learn to Mislead Humans via RLHFcited by 118
  11. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  12. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 101
  13. 2024Chain of Thoughtlessness? An Analysis of CoT in Planningcited by 100
  14. 2023Learning From Mistakes Makes LLM Better Reasonercited by 97
  15. 2023Evaluating Language Models for Mathematics through Interactionscited by 69
  16. 2023On Early Detection of Hallucinations in Factual Question Answeringcited by 54
  17. 2024Evaluating LLMs at Detecting Errors in LLM Responsescited by 49
  18. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  19. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 39
  20. 2021Learning to Give Checkable Answers with Prover-Verifier Gamescited by 29
  21. 2025Efficient Real-time Refinement of Language Model Text Generationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 31; 2026: 18.

49 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
  3. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  4. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
  5. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 52
  6. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Aheadcited by 43
  7. 2025When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoningcited by 41
  8. 2025Heimdall: test-time scaling on the generative verificationcited by 26
  9. 2025Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token Level Granularitycited by 22
  10. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 21
  11. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 18
  12. 2025Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifierscited by 14
37 more
  1. 2025GenSelect: A Generative Approach to Best-of-Ncited by 14
  2. 2025Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processingcited by 12
  3. 2025World Modelling Improves Language Model Agentscited by 12
  4. 2026V1: Unifying Generation and Self-Verification for Parallel Reasonerscited by 12
  5. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 11
  6. 2025Solve-Detect-Verify: Inference-Time Scaling with Flexible Generative Verifiercited by 7
  7. 2025Fact-Augmented Lookahead Planning for LLM Agentscited by 7
  8. 2025Test-time Scaling Techniques in Theoretical Physics -- A Comparison of Methods on the TPBench Datasetcited by 7
  9. 2025Unlocking Reasoning Capabilities in LLMs via Reinforcement Learning Explorationcited by 7
  10. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 5
  11. 2026TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulationcited by 4
  12. 2025Modeling Hierarchical Thinking in Large Reasoning Modelscited by 3
  13. 2025No Labels, No Problem: Training Visual Reasoners with Multimodal Verifierscited by 3
  14. 2026FUSE: Ensembling Verifiers with Zero Labeled Datacited by 3
  15. 2026Pseudo-Formalization for Automatic Proof Verificationcited by 3
  16. 2025Multiple Choice Learning of Low-Rank Adapters for Language Modelingcited by 2
  17. 2025Retrieval-augmented Decoding for Improving Truthfulness in Open-ended Generationcited by 2
  18. 2025Budget-aware Test-time Scaling via Discriminative Verificationcited by 2
  19. 2025Boosting Accuracy and Efficiency of Budget Forcing in LLMs via Reinforcement Learning for Mathematical Reasoningcited by 2
  20. 2026Learning Generative Selection for Best-of-Ncited by 2
  21. 2026LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoningcited by 2
  22. 2026Verify, Repair, Repeat, or Stop? Robust Stopping for Noisy Verify-Repair Loops in LLM Agentscited by 2
  23. 2025Bonsai: Interpretable Tree-Adaptive Grounded Reasoningcited by 1
  24. 2025Dist2ill: Distributional Distillation for One-Pass Uncertainty Estimation in Large Language Modelscited by 1
  25. 2025The Limits of Preference Data for Post-Trainingcited by 1
  26. 2026The Future of Facts: Tracing the Factual Generation-Verification Gapcited by 1
  27. 2026Difficulty-Adaptive Tree-Structured Policy Optimization for Expanding Reasoning Coverage in RLVRcited by 1
  28. 2025VeriMinder: Mitigating Analytical Vulnerabilities in NL2SQLcited by 0
  29. 2025Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrscited by 0
  30. 2026When LLMs Stop Following Steps: A Diagnostic Study of Procedural Execution in Language Modelscited by 0
  31. 2026Joint Consistency: A Unified Test-Time Aggregation Framework via Energy Minimizationcited by 0
  32. 2026DIRECT: When and Where Should You Allocate Test-Time Compute in Embodied Planners?cited by 0
  33. 2026Fork-Think with Confidencecited by 0
  34. 2026STEC: Evidence Compression for Deep Search in Open-domain Multi-Hop QAcited by 0
  35. 2026Claim-Level Reliability Assessment for Efficient Test-Time Reasoningcited by 0
  36. 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Taskscited by 0
  37. 2026How Much Can Language Models Gain from Test-Time Computation?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models
  2. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  3. 2025First Finish Search: Efficient Test-Time Scaling in Large Language Models
  4. 2025ARISE: An Adaptive Resolution-Aware Metric for Test-Time Scaling Evaluation in Large Reasoning Models
  5. 2025A*-Decoding: Token-Efficient Inference Scaling
  6. 2026What If We Allocate Test-Time Compute Adaptively?
  7. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoning
  8. 2025Trust but Verify! A Survey on Verification Design for Test-time Scaling