mlsanityData to 14 Oct 2026

Anchoring Bias in Large Language Models: An Experimental Study

arXiv 2412.06593 ↗First posted 9 Dec 2024cs.CLcited by 37

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate and comprehend human-like text. Despite their impressive capabilities, LLMs are not immune to limitations, including various biases. While much research has explored demographic biases, the cognitive biases in LLMs have not been equally scrutinized. This study delves into anchoring bias, a cognitive bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how anchoring bias manifests in LLMs and verify the effectiveness of various mitigation strategies. Our findings highlight the sensitivity of LLM responses to biased hints. At the same time, our experiments show that, to mitigate anchoring bias, one needs to collect hints from comprehensive angles to prevent the LLMs from being anchored to individual pieces of information, while simple algorithms such as Chain-of-Thought, Thoughts of Principles, Ignoring Anchor Hints, and Reflection are not sufficient.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 9 Dec 2024.

Builds on

13 of 28 references.

References · 13

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  3. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  4. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 159
  5. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  6. 2024Cognitive Bias in Decision-Making with LLMscited by 106
  7. 2024LLM economicus? Mapping the Behavioral Biases of LLMs via Utility Theorycited by 47
  8. 2024Addressing cognitive bias in medical language modelscited by 36
  9. 2024(Ir)rationality and Cognitive Biases in Large Language Modelscited by 23
  10. 2024AI Can Be Cognitively Biased: An Exploratory Study on Threshold Priming in LLM-Based Batch Relevance Assessmentcited by 20
  11. 2024Anchored Answers: Unravelling Positional Bias in GPT-2's Multiple-Choice Questionscited by 15
  12. 2023Evaluation of ChatGPT-Generated Medical Responses: A Systematic Review and Meta-Analysiscited by 13
1 more
  1. 2019Fooling with facts: Quantifying anchoring bias through a large-scale online experimentcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 25.

37 papers.

Sort:

By area:

  1. 2025TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Modelscited by 33
  2. 2026Large Language Model Reasoning Failurescited by 21
  3. 2025Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendationscited by 11
  4. 2025BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Modelscited by 11
  5. 2025How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulationscited by 7
  6. 2025Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigationscited by 6
  7. 2025Fragile Preferences: A Deep Dive Into Order Effects in Large Language Modelscited by 5
  8. 2026Cross-Context Review: Improving LLM Output Quality by Separating Production and Review Sessionscited by 5
  9. 2026Contextual Drag: How Errors in the Context Affect LLM Reasoningcited by 4
  10. 2025Revealing Political Bias in LLMs through Structured Multi-Agent Debatecited by 3
  11. 2025Anchors in the Machine: Behavioral and Attributional Evidence of Anchoring Bias in LLMscited by 3
  12. 2026Measuring and Exploiting Contextual Bias in LLM-Assisted Security Code Reviewcited by 3
25 more
  1. 2026AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMscited by 2
  2. 2025Getting out of the Big-Muddy: Escalation of Commitment in LLMscited by 1
  3. 2025A Role-Aware Multi-Agent Framework for Financial Education Question Answering with LLMscited by 1
  4. 2025Quantifying LLM Biases Across Instruction Boundary in Mixed Question Formscited by 1
  5. 2026LLM-as-a-judge validity is strongly task-dependent across physics assessment formatscited by 1
  6. 2026One Size Fits None: Heuristic Collapse in LLM Investment Advicecited by 1
  7. 2026Bias by Necessity: Impossibility Theorems for Sequential Processing with Convergent AI and Human Validationcited by 1
  8. 2026AMEL: Accumulated Message Effects on LLM Judgmentscited by 1
  9. 2026Machine Psychometrics: A Mathematical Psychology of Artificial Intelligencecited by 1
  10. 2025Fair Representation in Parliamentary Summaries: Measuring and Mitigating Inclusion Biascited by 0
  11. 2026Status Hierarchies in Language Modelscited by 0
  12. 2026"I followed what felt right, not what I was told": Autonomy, Coaching, and Recognizing Bias Through AI-Mediated Dialoguecited by 0
  13. 2026When Context Hurts: The Crossover Effect of Knowledge Transfer on Multi-Agent Design Explorationcited by 0
  14. 2026Don't Look at the Numbers: Visual Anchoring Bias and Layer-wise Representation in VLMscited by 0
  15. 2026Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulationscited by 0
  16. 2026Agentic Interpretation: Lattice-Structured Evidence for LLM-Based Program Analysiscited by 0
  17. 2026Automating Formal Verification with Agent-Guided Tree Searchcited by 0
  18. 2026Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perceptioncited by 0
  19. 2026Localizing Anchoring Pathways in Language Modelscited by 0
  20. 2026LLM Consumer Behavior Theory: Foundations of a Novel Research Fieldcited by 0
  21. 2026Prompt Framing Distorts Count Based Evaluation of LLM Error Detection: Evidence from Numeric Anchoringcited by 0
  22. 2026Confirming Our Biases? Evaluating the Capabilities, Risks, and Societal Impact of Large Language Modelscited by 0
  23. 2026The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systemscited by 0
  24. 2026Epistemic Warrant for LLM Recommendations: Characterizing the Basis for Reliance When Ground Truth Is Unavailablecited by 0
  25. 2026Measuring Human-Like Bias in LLMs? A Critique of Human-Derived Bias Constructs in LLM Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
  2. 2024CBEval: A framework for evaluating and interpreting cognitive biases in LLMs
  3. 2026Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
  4. 2024Interpreting Bias in Large Language Models: A Feature-Based Approach
  5. 2025Assessing Agentic Large Language Models in Multilingual National Bias
  6. 2025Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective
  7. 2024The Impact of Inference Acceleration on Bias of LLMs
  8. 2024Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation