mlsanityData to 7 Oct 2026

Strategic Self-Consistency

arXiv 2609.30352 ↗First posted 24 Sep 2026cs.LG cs.AI stat.MLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-consistency has become a popular technique for enhancing the reasoning abilities of large language models by generating multiple reasoning paths and selecting the final answer through a majority vote. However, because model providers typically charge users in proportion to the number of reasoning paths generated, they have a financial incentive to artificially increase the path count. In this work, we show that an unfaithful provider can exploit this incentive using a simple, efficient algorithm while avoiding detection by an auditor: by generating and strategically reordering additional reasoning paths, the algorithm makes every path appear necessary to reach the majority. To validate our algorithm, we conduct experiments with multiple instruct models from the Llama and Qwen families, as well as reasoning models distilled from DeepSeek-R1, on benchmark datasets spanning mathematics, science, and question answering. Our results suggest that the distribution of additional reasoning paths generated by our algorithm is heavy-tailed and that substantial capacity to overcharge remains even under the best possible audit designed to keep the false-positive rate below α = 0.1.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 46 of its 49 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

46 of 49 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026More Test-Time Compute Can Hurt: Overestimation Bias in LLM Beam Search16 Mar
  2. 2026Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning6 Jan
  3. 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models15 Nov
  4. 2025Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimation29 Jul
  5. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs19 May
  6. 2025Confidence Improves Self-Consistency in LLMs10 Feb
  7. 2024Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models18 Dec
  8. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning12 Dec
  9. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling30 Aug
  10. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
  11. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  12. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
1 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  6. 2024Tree Search for Language Model Agentscited by 148
  7. 2019Safe Testingcited by 98
  8. 2025Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignmentcited by 91
  9. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 84
  10. 2024Hypothesis testing with e-valuescited by 70
  11. 2024zkLLM: Zero Knowledge Proofs for Large Language Modelscited by 51
  12. 2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 35
21 more
  1. 2021Retrain or not retrain: Conformal test martingales for change-point detectioncited by 28
  2. 2023Fine-Tuning Games: Bargaining and Adaptation for General-Purpose Modelscited by 17
  3. 2025Auditing Black-Box LLM APIs with a Rank-Based Uniformity Testcited by 16
  4. 2025Is Your LLM Overcharging You? Tokenization, Transparency, and Incentivescited by 13
  5. 2024AuditLLM: A Tool for Auditing Large Language Models Using Multiprobe Approachcited by 11
  6. 2024Competition and Diversity in Generative AIcited by 10
  7. 2024Pricing and Competition for Generative AIcited by 8
  8. 2025Offline changepoint localization using a matrix of conformal p-valuescited by 8
  9. 2025AdaBoN: Adaptive Best-of-N Alignmentcited by 8
  10. 2026Optimal Bayesian Stopping for Efficient Inference of Consistent LLM Answerscited by 7
  11. 2026Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thoughtcited by 7
  12. 2025Robust ML Auditing using Prior Knowledgecited by 6
  13. 2025BEACON: Bayesian Optimal Stopping for Efficient LLM Samplingcited by 6
  14. 2021PAC Mode Estimation using PPR Martingale Confidence Sequencescited by 5
  15. 2025Auditing Pay-Per-Token in Large Language Modelscited by 4
  16. 2026Token-Efficient Change Detection in LLM APIscited by 4
  17. 2025Modeling the Economic Impacts of AI Openness Regulationcited by 3
  18. 2025Best-of-∞ -- Asymptotic Performance of Test-Time LLM Ensemblingcited by 3
  19. 2026Distribution-free changepoint localization after sequential change detectioncited by 3
  20. 2026Leaderboard Incentives: Model Rankings under Strategic Post-Trainingcited by 2
  21. 2026Computational Arbitrage in AI Model Marketscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  2. 2026Does Inference Scaling Improve Reasoning Faithfulness? A Multi-Model Analysis of Self-Consistency Tradeoffs
  3. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  4. 2026Self-Consistency via Marginal Sharpening
  5. 2026Improving reasoning at inference time via uncertainty minimisation
  6. 2023Universal Self-Consistency for Large Language Model Generation
  7. 2025Incentivizing LLMs to Self-Verify Their Answers
  8. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certainty