mlsanityData to 14 Oct 2026

Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs

arXiv 2510.17472 ↗First posted 20 Oct 2025stat.ML cs.LGcited by 12

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent advances such as self-consistency and test-time reinforcement learning (TTRL) improve the reliability of large language models (LLMs) without additional supervision, yet their underlying mechanisms and statistical guarantees remain poorly understood. We present a unified framework for certifiable inference in LLMs, showing that majority voting provides a statistical certificate of self-consistency: under mild assumptions, the aggregated answer coincides with the mode of the model's terminal distribution with high probability. We derive finite-sample and anytime-valid concentration bounds that quantify this confidence, and introduce the Martingale Majority Certificate (MMC), a sequential stopping rule that adaptively determines when sufficient samples have been drawn. We further prove that label-free post-training methods such as TTRL implicitly sharpen the answer distribution by exponentially tilting it toward its mode, thereby reducing the number of samples required for certification. Building on this insight, we propose new post-training objectives that explicitly optimise this trade-off between sharpness and bias. Together, these results explain and connect two central test-time scaling strategies, self-consistency and TTRL, within a single statistical framework for label-free, certifiable reliability in reasoning LLMs.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 55 of its 77 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

55 of 77 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Deep Think with Confidence21 Aug
  2. 2025Sampling-Efficient Test-Time Scaling: Self-Estimating the Best-of-N Sampling in Early Decoding3 Mar
  3. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty25 Feb
  4. 2025s1: Simple test-time scaling31 Jan
  5. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation3 Oct
  6. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling30 Aug
  7. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
  8. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  9. 2024Theoretical guarantees on the best-of-n alignment policy3 Jan
  10. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  12. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
2 more
  1. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar
  2. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  12. 2023Let's Verify Step by Stepcited by 3,789
29 more
  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  3. 2023PaLM 2 Technical Reportcited by 1,162
  4. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  5. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 886
  6. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  7. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  8. 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
  9. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166
  10. 2025General-Reasoner: Advancing LLM Reasoning Across All Domainscited by 135
  11. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 121
  12. 2024The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinismcited by 116
  13. 2025Maximizing Confidence Alone Improves Reasoningcited by 100
  14. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 96
  15. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 89
  16. 2024Hypothesis testing with e-valuescited by 74
  17. 2023EE-LLM: Large-Scale Training and Inference of Early-Exit Large Language Models with 3D Parallelismcited by 69
  18. 2025Can Large Reasoning Models Self-Train?cited by 68
  19. 2025Optimizing Language Models for Inference Time Objectives using Reinforcement Learningcited by 53
  20. 2024Self-Consistency Preference Optimizationcited by 50
  21. 2024Asymptotics of Language Model Alignmentcited by 47
  22. 2021On Tilted Losses in Machine Learning: Theory and Applicationscited by 43
  23. 2025IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theorycited by 40
  24. 2024Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanismcited by 32
  25. 2020Contextual Temperature for Language Modelingcited by 27
  26. 2025Hybrid Reinforcement: When Reward Is Sparse, It's Better to Be Densecited by 20
  27. 2025Unsupervised Elicitation of Language Modelscited by 11
  28. 2025Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directionscited by 11
  29. 2025Retrieval-augmented reasoning with lean language modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 12.

12 papers.

  1. 2026Beyond Consensus: Trace-Level Synthesis in Mixture of Agentscited by 4
  2. 2026ATLAS: Agentic Test-time Learning-to-Allocate Scalingcited by 4
  3. 2026MARS: Margin-Adversarial Risk-controlled Stopping for Parallel LLM Test-time Scalingcited by 3
  4. 2026Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibrationcited by 2
  5. 2026Beyond Majority Voting: Efficient Best-Of-N with Radial Consensus Scorecited by 2
  6. 2026CITE: Anytime-Valid Statistical Inference in LLM Self-Consistencycited by 2
  7. 2026When Does Delegation Beat Majority? A Delegation-Based Aggregator for Multi-Sample LLM Inferencecited by 1
  8. 2026HyPER: Bridging Exploration and Exploitation for Scalable LLM Reasoning with Hypothesis Path Expansion and Reductioncited by 0
  9. 2026A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliabilitycited by 0
  10. 2026Decomposing Wrong-Consensus Agreement in LLM Self-Consistencycited by 0
  11. 2026Provable Test-Time Scaling for Beam Search in LLM Reasoningcited by 0
  12. 2026Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
  2. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
  3. 2025Confidence Improves Self-Consistency in LLMs
  4. 2025Incentivizing LLMs to Self-Verify Their Answers
  5. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  6. 2026Self-CTRL: Self-Consistency Training with Reinforcement Learning
  7. 2025Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
  8. 2026Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning