# Addressing cognitive bias in medical language models

arXiv: https://arxiv.org/abs/2402.08113 · HTML: https://mlsanity.com/paper/2402.08113/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [MEDITRON-70B: Scaling Medical Pretraining for Large Language Models](https://mlsanity.com/paper/2311.16079/) (2023)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [PMC-LLaMA: Towards Building Open-source Language Models for Medicine](https://mlsanity.com/paper/2304.14454/) (2023)
- [Pathways: Asynchronous Distributed Dataflow for ML](https://mlsanity.com/paper/2203.12533/) (2022)
- [Medical Misinformation in AI-Assisted Self-Diagnosis: Development of a Method (EvalPrompt) for Analyzing Large Language Models](https://mlsanity.com/paper/2307.04910/) (2023)
- [Language models are susceptible to incorrect patient self-diagnosis in medical applications](https://mlsanity.com/paper/2309.09362/) (2023)
- [The Potential and Pitfalls of using a Large Language Model such as ChatGPT or GPT-4 as a Clinical Assistant](https://mlsanity.com/paper/2307.08152/) (2023)

## Built on by (most cited first, 30 of 36)

- [AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments](https://mlsanity.com/paper/2405.07960/) (2024)
- [A Survey of LLM-based Agents in Medicine: How far are we from Baymax?](https://mlsanity.com/paper/2502.11211/) (2025)
- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [Machine Unlearning in Generative AI: A Survey](https://mlsanity.com/paper/2407.20516/) (2024)
- [Anchoring Bias in Large Language Models: An Experimental Study](https://mlsanity.com/paper/2412.06593/) (2024)
- [A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions](https://mlsanity.com/paper/2406.03712/) (2024)
- [A Comprehensive Evaluation of Cognitive Biases in LLMs](https://mlsanity.com/paper/2410.15413/) (2024)
- [Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks](https://mlsanity.com/paper/2409.08087/) (2024)
- [General surgery vision transformer: A video pre-trained foundation model for general surgery](https://mlsanity.com/paper/2403.05949/) (2024)
- [Cognitive Biases in Large Language Models: A Survey and Mitigation Experiments](https://mlsanity.com/paper/2412.00323/) (2024)
- [GP-VLS: A general-purpose vision language model for surgery](https://mlsanity.com/paper/2407.19305/) (2024)
- [MedArabiQ: Benchmarking Large Language Models on Arabic Medical Tasks](https://mlsanity.com/paper/2505.03427/) (2025)
- [CURE-Med: Curriculum-Informed Reinforcement Learning for Multilingual Medical Reasoning](https://mlsanity.com/paper/2601.13262/) (2026)
- [From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practice](https://mlsanity.com/paper/2410.01812/) (2024)
- [Cognitive Biases in Large Language Models for News Recommendation](https://mlsanity.com/paper/2410.02897/) (2024)
- [Shallow Robustness, Deep Vulnerabilities: Multi-Turn Evaluation of Medical LLMs](https://mlsanity.com/paper/2510.12255/) (2025)
- [CLIMB: A Benchmark of Clinical Bias in Large Language Models](https://mlsanity.com/paper/2407.05250/) (2024)
- [Trustworthy Medical Question Answering: An Evaluation-Centric Survey](https://mlsanity.com/paper/2506.03659/) (2025)
- [Susceptibility of Large Language Models to User-Driven Factors in Medical Queries](https://mlsanity.com/paper/2503.22746/) (2025)
- [FairMedQA: Benchmarking Bias in Large Language Models for Medical Question Answering](https://mlsanity.com/paper/2505.19562/) (2025)
- [MindScope: Exploring cognitive biases in large language models through Multi-Agent Systems](https://mlsanity.com/paper/2410.04452/) (2024)
- [Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMs](https://mlsanity.com/paper/2507.07186/) (2025)
- [Self-Adaptive Cognitive Debiasing for Large Language Models in Decision-Making](https://mlsanity.com/paper/2504.04141/) (2025)
- [MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis](https://mlsanity.com/paper/2601.06636/) (2026)
- [Synth-SBDH: A Synthetic Dataset of Social and Behavioral Determinants of Health for Clinical Text](https://mlsanity.com/paper/2406.06056/) (2024)
- [Can Global XAI Methods Reveal Injected Behaviours in LLMs? SHAP vs Rule Extraction vs RuleSHAP](https://mlsanity.com/paper/2505.11189/) (2025)
- [Beyond Sliding Windows: Learning to Manage Memory in Non-Markovian Environments](https://mlsanity.com/paper/2512.19154/) (2025)
- [Information-seeking failures of large language models in agentic clinical reasoning](https://mlsanity.com/paper/2607.10275/) (2026)
- [Beyond Bias Scores: Unmasking Vacuous Neutrality in Small Language Models](https://mlsanity.com/paper/2506.08487/) (2025)
- [Toward Revealing Nuanced Biases in Medical LLMs](https://mlsanity.com/paper/2507.21176/) (2025)

## Abstract (arXiv)

There is increasing interest in the application large language models (LLMs) to the medical field, in part because of their impressive performance on medical exam questions. While promising, exam questions do not reflect the complexity of real patient-doctor interactions. In reality, physicians' decisions are shaped by many complex factors, such as patient compliance, personal experience, ethical beliefs, and cognitive bias. Taking a step toward understanding this, our hypothesis posits that when LLMs are confronted with clinical questions containing cognitive biases, they will yield significantly less accurate responses compared to the same questions presented without such biases. In this study, we developed BiasMedQA, a benchmark for evaluating cognitive biases in LLMs applied to medical tasks. Using BiasMedQA we evaluated six LLMs, namely GPT-4, Mixtral-8x70B, GPT-3.5, PaLM-2, Llama 2 70B-chat, and the medically specialized PMC Llama 13B. We tested these models on 1,273 questions from the US Medical Licensing Exam (USMLE) Steps 1, 2, and 3, modified to replicate common clinically-relevant cognitive biases. Our analysis revealed varying effects for biases on these LLMs, with GPT-4 standing out for its resilience to bias, in contrast to Llama 2 70B-chat and PMC Llama 13B, which were disproportionately affected by cognitive bias. Our findings highlight the critical need for bias mitigation in the development of medical LLMs, pointing towards safer and more reliable applications in healthcare.