# Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations

arXiv: https://arxiv.org/abs/2505.15392 · HTML: https://mlsanity.com/paper/2505.15392/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 21 May 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models](https://mlsanity.com/paper/2309.03883/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [Capturing Failures of Large Language Models via Human Cognitive Biases](https://mlsanity.com/paper/2202.12299/) (2022)
- [A Survey on Fairness in Large Language Models](https://mlsanity.com/paper/2308.10149/) (2023)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations](https://mlsanity.com/paper/2407.04069/) (2024)
- [From Tokens to Words: On the Inner Lexicon of LLMs](https://mlsanity.com/paper/2410.05864/) (2024)
- [Anchoring Bias in Large Language Models: An Experimental Study](https://mlsanity.com/paper/2412.06593/) (2024)
- [Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications](https://mlsanity.com/paper/2404.17196/) (2024)
- [Primacy Effect of ChatGPT](https://mlsanity.com/paper/2310.13206/) (2023)
- [CBEval: A framework for evaluating and interpreting cognitive biases in LLMs](https://mlsanity.com/paper/2412.03605/) (2024)
- [Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications](https://mlsanity.com/paper/2505.00049/) (2025)
- [Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-making](https://mlsanity.com/paper/2408.09176/) (2024)
- [Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inference](https://mlsanity.com/paper/2501.15754/) (2025)
- [Fooling with facts: Quantifying anchoring bias through a large-scale online experiment](https://mlsanity.com/paper/1911.12275/) (2019)

## Built on by (most cited first, 6 of 6)

- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [Contextual Drag: How Errors in the Context Affect LLM Reasoning](https://mlsanity.com/paper/2602.04288/) (2026)
- [Strategic Reflectivism In Intelligent Systems](https://mlsanity.com/paper/2505.22987/) (2025)
- [Assessing Cognitive Biases in LLMs for Judicial Decision Support: Virtuous Victim and Halo Effects](https://mlsanity.com/paper/2603.10016/) (2026)
- [Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulations](https://mlsanity.com/paper/2605.11789/) (2026)
- [The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systems](https://mlsanity.com/paper/2609.03425/) (2026)

## Abstract (arXiv)

The rise of Large Language Models (LLMs) like ChatGPT has advanced natural language processing, yet concerns about cognitive biases are growing. In this paper, we investigate the anchoring effect, a cognitive bias where the mind relies heavily on the first information as anchors to make affected judgments. We explore whether LLMs are affected by anchoring, the underlying mechanisms, and potential mitigation strategies. To facilitate studies at scale on the anchoring effect, we introduce a new dataset, SynAnchors (https://huggingface.co/datasets/TimTargaryen/SynAnchors). Combining refined evaluation metrics, we benchmark current widely used LLMs. Our findings show that LLMs' anchoring bias exists commonly with shallow-layer acting and can not be eliminated by conventional strategies, while reasoning can offer some mitigation.