# Existing LLMs Are Not Self-Consistent For Simple Tasks

arXiv: https://arxiv.org/abs/2506.18781 · HTML: https://mlsanity.com/paper/2506.18781/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 13 of 13)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Language Models Represent Space and Time](https://mlsanity.com/paper/2310.02207/) (2023)
- [Universal Self-Consistency for Large Language Model Generation](https://mlsanity.com/paper/2311.17311/) (2023)
- [On the Power of Foundation Models](https://mlsanity.com/paper/2211.16327/) (2022)
- [Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs](https://mlsanity.com/paper/2305.14279/) (2023)
- [Contrastive Learning Is Spectral Clustering On Similarity Graph](https://mlsanity.com/paper/2303.15103/) (2023)
- [Trade-off Between Efficiency and Consistency for Removal-based Explanations](https://mlsanity.com/paper/2210.17426/) (2022)
- [Reasoning about concepts with LLMs: Inconsistencies abound](https://mlsanity.com/paper/2405.20163/) (2024)
- [A Categorical Framework of General Intelligence](https://mlsanity.com/paper/2303.04571/) (2023)

## Built on by (most cited first, 5 of 5)

- [Direct Visual Grounding by Directing Attention of Visual Tokens](https://mlsanity.com/paper/2511.12738/) (2025)
- [Probing the Lack of Stable Internal Beliefs in LLMs](https://mlsanity.com/paper/2603.25187/) (2026)
- [Internal Planning in Language Models: Characterizing Horizon and Branch Awareness](https://mlsanity.com/paper/2509.25260/) (2025)
- [Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs](https://mlsanity.com/paper/2606.16011/) (2026)
- [Controlled Reformulation Testing for Logical Consistency in Large Language Models](https://mlsanity.com/paper/2607.14528/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have grown increasingly powerful, yet ensuring their decisions remain transparent and trustworthy requires self-consistency -- no contradictions in their internal reasoning. Our study reveals that even on simple tasks, such as comparing points on a line or a plane, or reasoning in a family tree, all smaller models are highly inconsistent, and even state-of-the-art models like DeepSeek-R1 and GPT-o4-mini are not fully self-consistent. To quantify and mitigate these inconsistencies, we introduce inconsistency metrics and propose two automated methods -- a graph-based and an energy-based approach. While these fixes provide partial improvements, they also highlight the complexity and importance of self-consistency in building more reliable and interpretable AI. The code and data are available at https://github.com/scorpio-nova/llm-self-consistency.