mlsanityData to 14 Oct 2026

Cognitive Bias in Decision-Making with LLMs

arXiv 2403.00811 ↗First posted 25 Feb 2024cs.AI cs.CLcited by 106

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) offer significant potential as tools to support an expanding range of decision-making tasks. Given their training on human (created) data, LLMs have been shown to inherit societal biases against protected groups, as well as be subject to bias functionally resembling cognitive bias. Human-like bias can impede fair and explainable decisions made with LLM assistance. Our work introduces BiasBuster, a framework designed to uncover, evaluate, and mitigate cognitive bias in LLMs, particularly in high-stakes decision-making tasks. Inspired by prior research in psychology and cognitive science, we develop a dataset containing 13,465 prompts to evaluate LLM decisions on different cognitive biases (e.g., prompt-induced, sequential, inherent). We test various bias mitigation strategies, while proposing a novel method utilizing LLMs to debias their own human-like cognitive bias within prompts. Our analysis provides a comprehensive picture of the presence and effects of cognitive bias across commercial and open-source models. We demonstrate that our selfhelp debiasing effectively mitigates model answers that display patterns akin to human cognitive bias without having to manually craft examples for each bias.

Key facts. Cited by 106 papers in the map (data through 14 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 25 Feb 2024.

Builds on

20 of 44 references.

References · 20

  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  2. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  3. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  4. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  6. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  7. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  8. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  9. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  10. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  11. 201850 Years of Test (Un)fairness: Lessons for Machine Learningcited by 101
  12. 2022Capturing Failures of Large Language Models via Human Cognitive Biasescited by 98
8 more
  1. 2023Prompted LLMs as Chatbot Modules for Long Open-domain Conversationcited by 61
  2. 2023Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Biascited by 53
  3. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  4. 2021Detect and Perturb: Neutral Rewriting of Biased and Sensitive Text via Gradient-based Decodingcited by 23
  5. 2023Challenging the appearance of machine intelligence: Cognitive bias in LLMs and Best Practices for Adoptioncited by 21
  6. 2022Despite "super-human" performance, current LLMs are unsuited for decisions about ethics and safetycited by 13
  7. 2021Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluationcited by 9
  8. 2023Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languagescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 47; 2026: 46.

106 papers.

Sort:

By area:

  1. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 62
  2. 2024Anchoring Bias in Large Language Models: An Experimental Studycited by 37
  3. 2025Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoningcited by 31
  4. 2026Large Language Model Reasoning Failurescited by 21
  5. 2024Cognitive Biases in Large Language Models: A Survey and Mitigation Experimentscited by 20
  6. 2025Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Modelscited by 20
  7. 2025Your AI, Not Your View: The Bias of LLMs in Investment Analysiscited by 17
  8. 2025Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinkingcited by 16
  9. 2025Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoningcited by 13
  10. 2024CBEval: A framework for evaluating and interpreting cognitive biases in LLMscited by 12
  11. 2025Bias Beware: The Impact of Cognitive Biases on LLM-Driven Product Recommendationscited by 11
  12. 2025From the Fluency Fallacy to the Micro-to-Macro Validity Gap: Opportunities and Pitfalls of LLMs in Social Simulationcited by 10
38 more
  1. 2025The Bias is in the Details: An Assessment of Cognitive Bias in LLMscited by 10
  2. 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMscited by 10
  3. 2024Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoptioncited by 9
  4. 2025FairCoder: Probing LLM Bias in High-Stakes Decision Making via Coding Taskscited by 9
  5. 2025Training an LLM-as-a-Judge Model: Pipeline, Insights, and Practical Lessonscited by 9
  6. 2025Detecting and Filtering Unsafe Training Data via Data Attribution with Denoised Representationcited by 9
  7. 2025CoBRA: Programming Cognitive Bias in Social Agents Using Classic Social Science Experimentscited by 8
  8. 2025What Makes a Good Natural Language Prompt?cited by 7
  9. 2025Quantifying Cognitive Bias Induction in LLM-Generated Contentcited by 7
  10. 2025How Does Cognitive Bias Affect Large Language Models? A Case Study on the Anchoring Effect in Price Negotiation Simulationscited by 7
  11. 2025Comparing the Framing Effect in Humans and LLMs on Naturally Occurring Textscited by 6
  12. 2025Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checkscited by 6
  13. 2025B-score: Detecting biases in large language models using response historycited by 6
  14. 2025BiasFilter: An Inference-Time Debiasing Framework for Large Language Modelscited by 6
  15. 2025Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Taskscited by 5
  16. 2025Fragile Preferences: A Deep Dive Into Order Effects in Large Language Modelscited by 5
  17. 2025Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMscited by 5
  18. 2025Acquiescence Bias in Large Language Modelscited by 5
  19. 2025Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoningcited by 5
  20. 2026When Agents See Humans as the Outgroup: Belief-Dependent Bias in LLM-Powered Agentscited by 5
  21. 2026Mechanisms of Prompt-Induced Hallucination in Vision-Language Modelscited by 5
  22. 2025Framing the Game: How Context Shapes LLM Decision-Makingcited by 4
  23. 2025Cognitive Agents Powered by Large Language Models for Agile Software Project Managementcited by 4
  24. 2026PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethicscited by 4
  25. 2026LLM Judges Can Be Too Generous When There Is No Reference Answercited by 4
  26. 2024Investigating Context Effects in Similarity Judgements in Large Language Modelscited by 3
  27. 2025Trapped by Expectations: Functional Fixedness in LLM-Enabled Chat Searchcited by 3
  28. 2025Toward Reasonable Parrots: Why Large Language Models Should Argue with Us by Designcited by 3
  29. 2025How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMscited by 3
  30. 2025Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Makingcited by 3
  31. 2026Label Effects: Shared Heuristic Reliance in Trust Assessment by Humans and LLM-as-a-Judgecited by 3
  32. 2026Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systemscited by 3
  33. 2024Balancing Rigor and Utility: Mitigating Cognitive Biases in Large Language Models for Multiple-Choice Questionscited by 2
  34. 2024Inertia in Moral and Value Judgments of Large Language Modelscited by 2
  35. 2025Talent or Luck? Evaluating Attribution Bias in Large Language Modelscited by 2
  36. 2025Is General-Purpose AI Reasoning Sensitive to Data-Induced Cognitive Biases? Dynamic Benchmarking on Typical Software Engineering Dilemmascited by 2
  37. 2025BiasFreeBench: a Benchmark for Mitigating Bias in Large Language Model Responsescited by 2
  38. 2025The Fragility of Strategic Thinking in Large Language Modelscited by 2

50 most cited of 106 All 106 (JSON)