mlsanityData to 14 Oct 2026

Estimating the Self-Consistency of LLMs

arXiv 2509.19489 ↗First posted 23 Sep 2025cs.AIcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Systems often repeat the same prompt to large language models (LLMs) and aggregate responses to improve reliability. This short note analyzes an estimator of the self-consistency of LLMs and the tradeoffs it induces under a fixed compute budget B=mn, where m is the number of prompts sampled from the task distribution and n is the number of repeated LLM calls per prompt; the resulting analysis favors a rough split m,npropto√B.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 3 of its 4 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

3 of 4 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Confidence Improves Self-Consistency in LLMs10 Feb

Other references · 2

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 148

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Two Calls, Two Moments, and the Vote-Accuracy Curve of Repeated LLM Inferencecited by 3
  2. 2026SFL-MTSC: Leveraging Semantic Frame-Level Multi-Task Self-Consistency for Robust Multi-Intent Spoken Language Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  2. 2023Self-Consistency of Large Language Models under Ambiguity
  3. 2026Position: It's Time to Optimize LLMs for Self-Consistency
  4. 2025Existing LLMs Are Not Self-Consistent For Simple Tasks
  5. 2026When Is Enough Enough in Self-Evolving LLM Systems?
  6. 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
  7. 2024Evaluating the Consistency of LLM Evaluators
  8. 2023Let's Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs