# Unveiling Confirmation Bias in Chain-of-Thought Reasoning

arXiv: https://arxiv.org/abs/2506.12301 · HTML: https://mlsanity.com/paper/2506.12301/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 14 Jun 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [What do we need to build explainable AI systems for the medical domain?](https://mlsanity.com/paper/1712.09923/) (2017)
- [Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse](https://mlsanity.com/paper/2410.21333/) (2024)
- [Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance](https://mlsanity.com/paper/2305.17306/) (2023)
- [Chain-of-Thought Unfaithfulness as Disguised Accuracy](https://mlsanity.com/paper/2402.14897/) (2024)
- [Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulness](https://mlsanity.com/paper/2405.18915/) (2024)

## Built on by (most cited first, 10 of 10)

- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Failing to Falsify: Evaluating and Mitigating Confirmation Bias in Language Models](https://mlsanity.com/paper/2604.02485/) (2026)
- [ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction](https://mlsanity.com/paper/2511.01188/) (2025)
- [Single LLM Debate, MoLaCE: Mixture of Latent Concept Experts Against Confirmation Bias](https://mlsanity.com/paper/2512.23518/) (2025)
- [D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models](https://mlsanity.com/paper/2601.17865/) (2026)
- [From Inertia to Objectivity: Improving Deep Research Agents with Noise Isolation](https://mlsanity.com/paper/2608.23045/) (2026)
- [Will Humanity Be Rendered Obsolete by AI?](https://mlsanity.com/paper/2510.22814/) (2025)
- [Humanity in the Age of AI: Reassessing 2025's Existential-Risk Narratives](https://mlsanity.com/paper/2512.04119/) (2025)
- [AILS-NTUA at SemEval-2026 Task 10: Agentic LLMs for Psycholinguistic Marker Extraction and Conspiracy Endorsement Detection](https://mlsanity.com/paper/2603.04921/) (2026)
- [When Verification Fails: How Compositionally Infeasible Claims Escape Rejection](https://mlsanity.com/paper/2604.10990/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) prompting has been widely adopted to enhance the reasoning capabilities of large language models (LLMs). However, the effectiveness of CoT reasoning is inconsistent across tasks with different reasoning types. This work presents a novel perspective to understand CoT behavior through the lens of textitconfirmation bias in cognitive psychology. Specifically, we examine how model internal beliefs, approximated by direct question-answering probabilities, affect both reasoning generation (Q → R) and reasoning-guided answer prediction (QR → A) in CoT. By decomposing CoT into a two-stage process, we conduct a thorough correlation analysis in model beliefs, rationale attributes, and stage-wise performance. Our results provide strong evidence of confirmation bias in LLMs, such that model beliefs not only skew the reasoning process but also influence how rationales are utilized for answer prediction. Furthermore, the interplay between task vulnerability to confirmation bias and the strength of beliefs also provides explanations for CoT effectiveness across reasoning tasks and models. Overall, this study provides a valuable insight for the needs of better prompting strategies that mitigate confirmation bias to enhance reasoning performance. Code is available at textithttps://github.com/yuewan2/biasedcot.