mlsanityData to 14 Oct 2026

Existing LLMs Are Not Self-Consistent For Simple Tasks

arXiv 2506.18781 ↗First posted 23 Jun 2025cs.CLcited by 5

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Language Models (LLMs) have grown increasingly powerful, yet ensuring their decisions remain transparent and trustworthy requires self-consistency -- no contradictions in their internal reasoning. Our study reveals that even on simple tasks, such as comparing points on a line or a plane, or reasoning in a family tree, all smaller models are highly inconsistent, and even state-of-the-art models like DeepSeek-R1 and GPT-o4-mini are not fully self-consistent. To quantify and mitigate these inconsistencies, we introduce inconsistency metrics and propose two automated methods -- a graph-based and an energy-based approach. While these fixes provide partial improvements, they also highlight the complexity and importance of self-consistency in building more reliable and interpretable AI. The code and data are available at https://github.com/scorpio-nova/llm-self-consistency.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 13 of its 19 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

13 of 19 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Universal Self-Consistency for Large Language Model Generation29 Nov
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  3. 2023Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs23 May

Other references · 10

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  5. 2023Language Models Represent Space and Timecited by 316
  6. 2022On the Power of Foundation Modelscited by 37
  7. 2023Contrastive Learning Is Spectral Clustering On Similarity Graphcited by 25
  8. 2022Trade-off Between Efficiency and Consistency for Removal-based Explanationscited by 6
  9. 2024Reasoning about concepts with LLMs: Inconsistencies aboundcited by 6
  10. 2023A Categorical Framework of General Intelligencecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025Direct Visual Grounding by Directing Attention of Visual Tokenscited by 7
  2. 2026Probing the Lack of Stable Internal Beliefs in LLMscited by 2
  3. 2025Internal Planning in Language Models: Characterizing Horizon and Branch Awarenesscited by 1
  4. 2026Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMscited by 1
  5. 2026Controlled Reformulation Testing for Logical Consistency in Large Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Self-Consistency Is Losing Its Edge: Diminishing Returns and Rising Costs in Modern LLMs
  2. 2024Semantic Self-Consistency: Enhancing Language Model Reasoning via Semantic Weighting
  3. 2023Self-Consistency of Large Language Models under Ambiguity
  4. 2026The Consistency Dilemma in LLMs: Generator-Evaluator Agreement and Vulnerability to Mistakes
  5. 2026Position: It's Time to Optimize LLMs for Self-Consistency
  6. 2024Lachesis: Predicting LLM Inference Accuracy using Structural Properties of Reasoning Paths
  7. 2024Internal Consistency and Self-Feedback in Large Language Models: A Survey
  8. 2025Bridging Internal Probability and Self-Consistency for Effective and Efficient LLM Reasoning