mlsanityData to 7 Oct 2026

Scalable Best-of-N Selection for Large Language Models via Self-Certainty

arXiv 2502.18581 ↗First posted 25 Feb 2025cs.CL cs.AI cs.LGcited by 182

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models (LLMs) through increased test-time computation. Current state-of-the-art methods often employ computationally intensive reward models for response evaluation and selection. Reward-free alternatives, like self-consistency and universal self-consistency, are limited in their ability to handle open-ended generation tasks or scale effectively. To address these limitations, we propose self-certainty, a novel and efficient metric that leverages the inherent probability distribution of LLM outputs to estimate response quality without requiring external reward models. We hypothesize that higher distributional self-certainty, aggregated across multiple samples, correlates with improved response accuracy, as it reflects greater confidence in the generated output. Through extensive experiments on various reasoning tasks, we demonstrate that self-certainty (1) scales effectively with increasing sample size N, akin to reward models but without the computational overhead; (2) complements chain-of-thought, improving reasoning performance beyond greedy decoding; and (3) generalizes to open-ended tasks where traditional self-consistency methods fall short. Our findings establish self-certainty as a practical and efficient way for improving LLM reasoning capabilities. The code is available at https://github.com/backprop07/Self-Certainty

Key facts. Cited by 182 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 36 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

27 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  2. 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey19 Jul
  3. 2023Universal Self-Consistency for Large Language Model Generation29 Nov

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  11. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  12. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
12 more
  1. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  2. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  3. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  4. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  5. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 225
  6. 2023Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hackingcited by 195
  7. 2023Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthinesscited by 121
  8. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 108
  9. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 94
  10. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 74
  11. 2024Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?cited by 45
  12. 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 63; 2026: 119.

182 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  3. 2025Learning to Reason without External Rewardscited by 199
  4. 2025Deep Think with Confidencecited by 199
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  6. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  7. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  8. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 49
  9. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 34
  10. 2025Unlocking Exploration in RLVR: Uncertainty-aware Advantage Shaping for Deeper Reasoningcited by 34
  11. 2025AlphaOne: Reasoning Models Thinking Slow and Fast at Test Timecited by 27
  12. 2025Reinforcing Video Reasoning with Focused Thinkingcited by 26
38 more
  1. 2025TokUR: Token-Level Uncertainty Estimation for Large Language Model Reasoningcited by 24
  2. 2025Verifier-free Test-Time Sampling for Vision-Language-Action Modelscited by 21
  3. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokenscited by 21
  4. 2025Shop-R1: Rewarding LLMs to Simulate Human Behavior in Online Shopping via Reinforcement Learningcited by 19
  5. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  6. 2025Let's (not) just put things in Context: Test-Time Training for Long-Context LLMscited by 17
  7. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 15
  8. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 14
  9. 2025Know When to Explore: Difficulty-Aware Certainty as a Guide for LLM Reinforcement Learningcited by 12
  10. 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMscited by 12
  11. 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifiercited by 12
  12. 2025From Efficiency to Adaptivity: A Deeper Look at Adaptive Reasoning in Large Language Modelscited by 11
  13. 2026Accurate Failure Prediction in Agents Does Not Imply Effective Failure Preventioncited by 11
  14. 2026Scaling Test-Time Compute for Agentic Codingcited by 11
  15. 2025Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Frameworkcited by 10
  16. 2025EAGer: Entropy-Aware GEneRation for Adaptive Inference-Time Scalingcited by 10
  17. 2025Harnessing Consistency for Robust Test-Time LLM Ensemblecited by 10
  18. 2025Do LLMs Signal When They're Right? Evidence from Neuron Agreementcited by 10
  19. 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoningcited by 10
  20. 2026Hidden States as Early Signals: Step-level Trace Evaluation and Pruning for Efficient Test-Time Scalingcited by 10
  21. 2026EDIS: Diagnosing LLM Reasoning via Entropy Dynamicscited by 10
  22. 2025PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chainscited by 9
  23. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  24. 2026Do Not Waste Your Rollouts: Recycling Search Experience for Efficient Test-Time Scalingcited by 9
  25. 2026Complementing Self-Consistency with Cross-Model Disagreement for Uncertainty Quantificationcited by 9
  26. 2025Batched Self-Consistency Improves LLM Relevance Assessment and Rankingcited by 8
  27. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 8
  28. 2026Prune as You Generate: Online Rollout Pruning for Faster and Better RLVRcited by 8
  29. 2025DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Samplingcited by 7
  30. 2025Uncertainty-Aware Answer Selection for Improved Reasoning in Multi-LLM Systemscited by 6
  31. 2025Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimizationcited by 6
  32. 2025Klear-AgentForge: Forging Agentic Intelligence through Posttraining Scalingcited by 6
  33. 2025Self-Evaluating LLMs for Multi-Step Tasks: Stepwise Confidence Estimation for Failure Detectioncited by 6
  34. 2025SSR: Socratic Self-Refine for Large Language Model Reasoningcited by 6
  35. 2025On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inferencecited by 6
  36. 2026Multi-Token Prediction via Self-Distillationcited by 6
  37. 2026Self-Refining Vision Language Model for Robotic Failure Detection and Reasoningcited by 6
  38. 2026TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoningcited by 6

50 most cited of 182 All 182 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Enhancing reasoning accuracy in large language models during inference time
  2. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  3. 2025Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
  4. 2025Self-Correcting Large Language Models: Generation vs. Multiple Choice
  5. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
  6. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling
  7. 2025Self-Training Large Language Models with Confident Reasoning
  8. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting