Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
On the map: LLM reasoning → Test-time compute and efficient thinking
Abstract (arXiv)
Self-consistency -- sampling multiple reasoning paths and selecting the most frequent answer -- was designed for an era when language models made frequent, unpredictable errors. This study argues that the technique has become increasingly wasteful as models grow stronger, and may degrade performance on problems that modern models already solve reliably. Using Gemini 2.5 models on HotpotQA and MATH-500, we show that accuracy gains from increasing the number of sampled reasoning paths are minimal -- 0.4% on HotpotQA across 20 samples, and 1.6% on MATH-500 -- while token costs scale nearly linearly with sample count. Critically, performance plateaued early and in some configurations declined at high sample counts, suggesting that additional paths introduce noise rather than signal when models already solve problems reliably. As inference costs rise with model scale, indiscriminate self-consistency is difficult to justify. We recommend reserving multi-path sampling for problems that demonstrably exceed a model's single-pass reliability.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 6 of its 8 references that are in the map. First posted on arXiv 2 Nov 2025.
Builds on
6 of 8 references.
Closest ancestors · 2
Same direction, newest first
- 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling30 Aug
- 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs19 May
Other references · 4
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning
- 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs
- 2025Existing LLMs Are Not Self-Consistent For Simple Tasks
- 2025Optimal Self-Consistency for Efficient Reasoning with Large Language Models
- 2024Path-Consistency with Prefix Enhancement for Efficient Inference in LLMs
- 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
- 2026Does Inference Scaling Improve Reasoning Faithfulness? A Multi-Model Analysis of Self-Consistency Tradeoffs
- 2026Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning