# Large Language Models are overconfident and amplify human bias

arXiv: https://arxiv.org/abs/2505.02151 · HTML: https://mlsanity.com/paper/2505.02151/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2025.

## Built on by (most cited first, 30 of 30)

- [Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution](https://mlsanity.com/paper/2505.20286/) (2025)
- [Large language models provide unsafe answers to patient-posed medical questions](https://mlsanity.com/paper/2507.18905/) (2025)
- [Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missing](https://mlsanity.com/paper/2507.10587/) (2025)
- [Calibrating Verbalized Confidence with Self-Generated Distractors](https://mlsanity.com/paper/2509.25532/) (2025)
- [ThinkQE: Query Expansion via an Evolving Thinking Process](https://mlsanity.com/paper/2506.09260/) (2025)
- [Do Large Language Models Know What They Are Capable Of?](https://mlsanity.com/paper/2512.24661/) (2025)
- [Agentic Uncertainty Reveals Agentic Overconfidence](https://mlsanity.com/paper/2602.06948/) (2026)
- [Can Large Language Models Match the Conclusions of Systematic Reviews?](https://mlsanity.com/paper/2505.22787/) (2025)
- [KnowGuard: Knowledge-Driven Abstention for Multi-Round Clinical Reasoning](https://mlsanity.com/paper/2509.24816/) (2025)
- [CaRT: Teaching LLM Agents to Know When They Know Enough](https://mlsanity.com/paper/2510.08517/) (2025)
- [Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models](https://mlsanity.com/paper/2506.14092/) (2025)
- [A Collectivist, Economic Perspective on AI](https://mlsanity.com/paper/2507.06268/) (2025)
- [A word association network methodology for evaluating implicit biases in LLMs compared to humans](https://mlsanity.com/paper/2510.24488/) (2025)
- [Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment](https://mlsanity.com/paper/2511.13290/) (2025)
- [A Cross-Lingual Analysis of Bias in Large Language Models Using Romanian History](https://mlsanity.com/paper/2510.02362/) (2025)
- [Completing Missing Annotation: Multi-Agent Debate for Accurate and Scalable Relevant Assessment for IR Benchmarks](https://mlsanity.com/paper/2602.06526/) (2026)
- [Bayesian Elicitation with LLMs: Model Size Helps, Extra "Reasoning" Doesn't Always](https://mlsanity.com/paper/2604.01896/) (2026)
- [ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Models](https://mlsanity.com/paper/2605.10328/) (2026)
- [Self-Distilled Trajectory-Aware Boltzmann Modeling: Bridging the Training-Inference Discrepancy in Diffusion Language Models](https://mlsanity.com/paper/2605.11854/) (2026)
- [Capability Self-Assessment in Large Language Models](https://mlsanity.com/paper/2606.00251/) (2026)
- [A Context-Aware Dual-Metric Framework for Confidence Estimation in Large Language Models](https://mlsanity.com/paper/2508.00600/) (2025)
- [Does In-IDE Calibration of Large Language Models work at Scale?](https://mlsanity.com/paper/2510.22614/) (2025)
- [The New Pro Se: Generative AI and the Surge in Federal Civil Self-Representation](https://mlsanity.com/paper/2605.29493/) (2026)
- [Uncertainty-Aware LLM-Guided Policy Shaping for Sparse-Reward Reinforcement Learning](https://mlsanity.com/paper/2606.06673/) (2026)
- [What Do People Actually Want From AI? Mapping Preference Plurality](https://mlsanity.com/paper/2606.06674/) (2026)
- [Co-Construction Blindness and Asymmetric Epistemic Vulnerability in Human-LLM Interaction](https://mlsanity.com/paper/2606.20762/) (2026)
- [Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs](https://mlsanity.com/paper/2608.05660/) (2026)
- [CWM: Controllable White-Box Meta-Prompting for Adaptive Retrieval-Augmented Generation and Reasoning Ability](https://mlsanity.com/paper/2609.15234/) (2026)
- [LLMs learn different forms of metacognition when trained to predict their own accuracy](https://mlsanity.com/paper/2609.33886/) (2026)
- [Distilling What Matters: Confidence-Aware Selective Distillation for Large Language Models](https://mlsanity.com/paper/2609.36734/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are revolutionizing every aspect of society. They are increasingly used in problem-solving tasks to substitute human assessment and reasoning. LLMs are trained on what humans write and are thus exposed to human bias. We evaluate whether LLMs inherit one of the most widespread human biases: overconfidence. We algorithmically construct reasoning problems with known ground truths. We prompt LLMs to answer these problems and assess the confidence in their answers, closely following similar protocols in human experiments. We find that all five LLMs we study are overconfident: they overestimate the probability that their answer is correct between 20% and 60%. Humans have accuracy similar to the more advanced LLMs, but far lower overconfidence. Although humans and LLMs are similarly biased in questions which they are certain they answered correctly, a key difference emerges between them: LLM bias increases sharply relative to humans if they become less sure that their answers are correct. We also show that LLM input has ambiguous effects on human decision making: LLM input leads to an increase in the accuracy, but it more than doubles the extent of overconfidence in the answers.