# How Overconfidence in Initial Choices and Underconfidence Under Criticism Modulate Change of Mind in Large Language Models

arXiv: https://arxiv.org/abs/2507.03120 · HTML: https://mlsanity.com/paper/2507.03120/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 3 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [Measuring short-form factuality in large language models](https://mlsanity.com/paper/2411.04368/) (2024)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [Accounting for Sycophancy in Language Model Uncertainty Estimation](https://mlsanity.com/paper/2410.14746/) (2024)
- [Two Sides of Miscalibration: Identifying Over and Under-Confidence Prediction for Network Calibration](https://mlsanity.com/paper/2308.03172/) (2023)

## Built on by (most cited first, 9 of 9)

- [Technological folie `a deux: Feedback Loops Between AI Chatbots and Mental Illness](https://mlsanity.com/paper/2507.19218/) (2025)
- [Evidence for Limited Metacognition in LLMs](https://mlsanity.com/paper/2509.21545/) (2025)
- [Evaluating LLMs in Finance Requires Explicit Bias Consideration](https://mlsanity.com/paper/2602.14233/) (2026)
- [Causal Evidence that Language Models use Confidence to Drive Behavior](https://mlsanity.com/paper/2603.22161/) (2026)
- [KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning](https://mlsanity.com/paper/2509.24816/) (2025)
- [Monitor-Generate-Verify (MGV): Formalising Metacognitive Theory for Language Model Reasoning](https://mlsanity.com/paper/2511.04341/) (2025)
- [Learning from Self-Debate: Preparing Reasoning Models for Multi-Agent Debate](https://mlsanity.com/paper/2601.22297/) (2026)
- [What Am I Missing? Question-Answering as Hidden State Probing](https://mlsanity.com/paper/2605.31561/) (2026)
- [Aligning to Illusions: Choice Blindness in Human and AI Feedback](https://mlsanity.com/paper/2603.08412/) (2026)

## Abstract (arXiv)

Large language models (LLMs) exhibit strikingly conflicting behaviors: they can appear steadfastly overconfident in their initial answers whilst at the same time being prone to excessive doubt when challenged. To investigate this apparent paradox, we developed a novel experimental paradigm, exploiting the unique ability to obtain confidence estimates from LLMs without creating memory of their initial judgments -- something impossible in human participants. We show that LLMs -- Gemma 3, GPT4o and o1-preview -- exhibit a pronounced choice-supportive bias that reinforces and boosts their estimate of confidence in their answer, resulting in a marked resistance to change their mind. We further demonstrate that LLMs markedly overweight inconsistent compared to consistent advice, in a fashion that deviates qualitatively from normative Bayesian updating. Finally, we demonstrate that these two mechanisms -- a drive to maintain consistency with prior commitments and hypersensitivity to contradictory feedback -- parsimoniously capture LLM behavior in a different domain. Together, these findings furnish a mechanistic account of LLM confidence that explains both their stubbornness and excessive sensitivity to criticism.