Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Self-consistency samples many reasoning trajectories and aggregates their final answers, treating the LLM as a black box that returns one answer per trajectory. Yet the final answer of each trajectory is sampled from a softmax vector that is available from the model's log-probabilities. We refer to this as the grey-box setting in which each trajectory reveals this answer distribution rather than a single draw from it. We formulate efficient inference in this setting as sequential mode identification with distribution-valued observations: sample trajectories one at a time and stop as soon as the LLM's modal answer is identified at a prescribed confidence level. We characterize the asymptotic stopping rate of mode identification with distribution-valued observations exactly and show that it is never worse than the black-box rate. We then propose the ASC-D algorithm, a betting stopping rule that attains this asymptotic stopping rate. On MMLU-Redux, ASC-D uses 46.4--95.6% fewer trajectories than answer-only adaptive self-consistency baselines and achieves the highest fixed-budget correct-certification rate across three open-source models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 23 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
18 of 23 references.
Closest ancestors · 9
Same direction, newest first
- 2026CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency7 May
- 2025Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs20 Oct
- 2025Deep Think with Confidence21 Aug
- 2025Sample Complexity and Representation Ability of Test-time Scaling Paradigms5 Jun
- 2025Confidence Improves Self-Consistency in LLMs10 Feb
- 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling30 Aug
- 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
- 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
- 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
Other references · 9
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2014On the Complexity of Best Arm Identification in Multi-Armed Bandit Modelscited by 280
- 2016Optimal Best Arm Identification with Fixed Confidencecited by 233
- 2026Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answerscited by 7
- 2019Sequential Mode Estimation with Oracle Queriescited by 5
- 2021PAC Mode Estimation using PPR Martingale Confidence Sequencescited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
- 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation
- 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning
- 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
- 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
- 2026Self-Consistency via Marginal Sharpening
- 2025CGES: Confidence-Guided Early Stopping for Efficient and Accurate Self-Consistency
- 2026Quantifying Consistency in LLM Logical Reasoning via Structural Uncertainty