# Anchoring Bias in Large Language Models: An Experimental Study

arXiv: https://arxiv.org/abs/2412.06593 · HTML: https://mlsanity.com/paper/2412.06593/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 9 Dec 2024.

## Builds on (most cited first, 13 of 13)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models](https://mlsanity.com/paper/2310.06117/) (2023)
- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theory](https://mlsanity.com/paper/2408.02784/) (2024)
- [Addressing cognitive bias in medical language models](https://mlsanity.com/paper/2402.08113/) (2024)
- [(Ir)rationality and Cognitive Biases in Large Language Models](https://mlsanity.com/paper/2402.09193/) (2024)
- [AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessment](https://mlsanity.com/paper/2409.16022/) (2024)
- [Anchored Answers: Unravelling Positional Bias in GPT-2's Multiple-Choice Questions](https://mlsanity.com/paper/2405.03205/) (2024)
- [Evaluation of ChatGPT-Generated Medical Responses: A Systematic Review and Meta-Analysis](https://mlsanity.com/paper/2310.08410/) (2023)
- [Fooling with facts: Quantifying anchoring bias through a large-scale online experiment](https://mlsanity.com/paper/1911.12275/) (2019)

## Built on by (most cited first, 30 of 37)

- [TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models](https://mlsanity.com/paper/2503.11656/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendations](https://mlsanity.com/paper/2502.01349/) (2025)
- [BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models](https://mlsanity.com/paper/2510.00307/) (2025)
- [How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulations](https://mlsanity.com/paper/2508.21137/) (2025)
- [Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations](https://mlsanity.com/paper/2505.15392/) (2025)
- [Fragile Preferences: A Deep Dive Into Order Effects in Large Language Models](https://mlsanity.com/paper/2506.14092/) (2025)
- [Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessions](https://mlsanity.com/paper/2603.12123/) (2026)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [Revealing Political Bias in LLMs through Structured Multi-Agent Debate](https://mlsanity.com/paper/2506.11825/) (2025)
- [Anchors in the Machine: Behavioral and Attributional Evidence of Anchoring Bias in LLMs](https://mlsanity.com/paper/2511.05766/) (2025)
- [Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Review](https://mlsanity.com/paper/2603.18740/) (2026)
- [AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs](https://mlsanity.com/paper/2608.14320/) (2026)
- [Getting out of the Big-Muddy: Escalation of Commitment in LLMs](https://mlsanity.com/paper/2508.01545/) (2025)
- [A Role-Aware Multi-Agent Framework for Financial Education Question Answering with LLMs](https://mlsanity.com/paper/2509.09727/) (2025)
- [Quantifying LLM Biases Across Instruction Boundary in Mixed Question Forms](https://mlsanity.com/paper/2509.20278/) (2025)
- [LLM-as-a-judge validity is strongly task-dependent across physics assessment formats](https://mlsanity.com/paper/2603.14732/) (2026)
- [One Size Fits None: Heuristic Collapse in LLM Investment Advice](https://mlsanity.com/paper/2604.23837/) (2026)
- [Bias by Necessity: Impossibility Theorems for Sequential Processing with Convergent AI and Human Validation](https://mlsanity.com/paper/2605.08716/) (2026)
- [AMEL: Accumulated Message Effects on LLM Judgments](https://mlsanity.com/paper/2605.22714/) (2026)
- [Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence](https://mlsanity.com/paper/2605.23952/) (2026)
- [Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Bias](https://mlsanity.com/paper/2507.14221/) (2025)
- [Status Hierarchies in Language Models](https://mlsanity.com/paper/2601.17577/) (2026)
- ["I followed what felt right, not what I was told": Autonomy, Coaching, and Recognizing Bias Through AI-Mediated Dialogue](https://mlsanity.com/paper/2603.11274/) (2026)
- [When Context Hurts: The Crossover Effect of Knowledge Transfer on Multi-Agent Design Exploration](https://mlsanity.com/paper/2605.04361/) (2026)
- [Don't Look at the Numbers: Visual Anchoring Bias and Layer-wise Representation in VLMs](https://mlsanity.com/paper/2605.11218/) (2026)
- [Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulations](https://mlsanity.com/paper/2605.11789/) (2026)
- [Agentic Interpretation: Lattice-Structured Evidence for LLM-Based Program Analysis](https://mlsanity.com/paper/2605.12694/) (2026)
- [Automating Formal Verification with Agent-Guided Tree Search](https://mlsanity.com/paper/2605.27485/) (2026)
- [Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception](https://mlsanity.com/paper/2606.06714/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate and comprehend human-like text. Despite their impressive capabilities, LLMs are not immune to limitations, including various biases. While much research has explored demographic biases, the cognitive biases in LLMs have not been equally scrutinized. This study delves into anchoring bias, a cognitive bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how anchoring bias manifests in LLMs and verify the effectiveness of various mitigation strategies. Our findings highlight the sensitivity of LLM responses to biased hints. At the same time, our experiments show that, to mitigate anchoring bias, one needs to collect hints from comprehensive angles to prevent the LLMs from being anchored to individual pieces of information, while simple algorithms such as Chain-of-Thought, Thoughts of Principles, Ignoring Anchor Hints, and Reflection are not sufficient.