mlsanityData to 14 Oct 2026

Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigations

arXiv 2505.15392 ↗First posted 21 May 2025cs.CLcited by 6

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

The rise of Large Language Models (LLMs) like ChatGPT has advanced natural language processing, yet concerns about cognitive biases are growing. In this paper, we investigate the anchoring effect, a cognitive bias where the mind relies heavily on the first information as anchors to make affected judgments. We explore whether LLMs are affected by anchoring, the underlying mechanisms, and potential mitigation strategies. To facilitate studies at scale on the anchoring effect, we introduce a new dataset, SynAnchors (https://huggingface.co/datasets/TimTargaryen/SynAnchors). Combining refined evaluation metrics, we benchmark current widely used LLMs. Our findings show that LLMs' anchoring bias exists commonly with shallow-layer acting and can not be eliminated by conventional strategies, while reasoning can offer some mitigation.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 26 of its 50 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

26 of 50 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  11. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  12. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
14 more
  1. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  2. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
  3. 2023A Survey on Fairness in Large Language Modelscited by 95
  4. 2021Trustworthy AI: A Computational Perspectivecited by 93
  5. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 85
  6. 2024From Tokens to Words: On the Inner Lexicon of LLMscited by 48
  7. 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
  8. 2024Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applicationscited by 33
  9. 2023Primacy Effect of ChatGPTcited by 27
  10. 2024CBEval: A framework for evaluating and interpreting cognitive biases in LLMscited by 12
  11. 2025Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applicationscited by 10
  12. 2024Cognitive LLMs: Towards Integrating Cognitive Architectures and Large Language Models for Manufacturing Decision-makingcited by 5
  13. 2025Weight-based Analysis of Detokenization in Language Models: Understanding the First Stage of Inference Without Inferencecited by 5
  14. 2019Fooling with facts: Quantifying anchoring bias through a large-scale online experimentcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026Large Language Model Reasoning Failurescited by 21
  2. 2026Contextual Drag: How Errors in the Context Affect LLM Reasoningcited by 4
  3. 2025Strategic Reflectivism In Intelligent Systemscited by 0
  4. 2026Assessing Cognitive Biases in LLMs for Judicial Decision Support: Virtuous Victim and Halo Effectscited by 0
  5. 2026Beyond Inefficiency: Systemic Costs of Incivility in Multi-Agent Monte Carlo Simulationscited by 0
  6. 2026The Civilization Framework: Sovereign-Anchored Communication Between Personal Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs
  2. 2025Anchors in the Machine: Behavioral and Attributional Evidence of Anchoring Bias in LLMs
  3. 2025How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulations
  4. 2026Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence
  5. 2026CogBias: Measuring and Mitigating Cognitive Bias in Large Language Models
  6. 2025Positional Biases Shift as Inputs Approach Context Window Limits
  7. 2026The Evidence Anchoring Boundary: Where Visual Interventions Matter and When They Help in Multimodal LLMs
  8. 2025Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs