# Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation

arXiv: https://arxiv.org/abs/2401.13867 · HTML: https://mlsanity.com/paper/2401.13867/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 7 of 7)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Capabilities of GPT-4 on Medical Challenge Problems](https://mlsanity.com/paper/2303.13375/) (2023)
- [Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Health](https://mlsanity.com/paper/2306.10070/) (2023)
- [Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicity](https://mlsanity.com/paper/2301.12867/) (2023)
- [Matching Patients to Clinical Trials with Large Language Models](https://mlsanity.com/paper/2307.15051/) (2023)
- [PMC-Patients: A Large-scale Dataset of Patient Summaries and Relations for Benchmarking Retrieval-based Clinical Decision Support Systems](https://mlsanity.com/paper/2202.13876/) (2022)

## Built on by (most cited first, 30 of 31)

- [A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine](https://mlsanity.com/paper/2405.08603/) (2024)
- [A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare](https://mlsanity.com/paper/2502.15871/) (2025)
- [FIRE-Bench: Evaluating AI Agents on the Rediscovery of Scientific Insights](https://mlsanity.com/paper/2602.02905/) (2026)
- [Multimodal Large Language Models for Medicine: A Comprehensive Survey](https://mlsanity.com/paper/2504.21051/) (2025)
- [DAIQ: Auditing Demographic Attribute Inference from Question in LLMs](https://mlsanity.com/paper/2508.15830/) (2025)
- [A survey on fairness of large language models in e-commerce: progress, application, and challenge](https://mlsanity.com/paper/2405.13025/) (2024)
- [CLIMB: A Benchmark of Clinical Bias in Large Language Models](https://mlsanity.com/paper/2407.05250/) (2024)
- [Elucidating Mechanisms of Demographic Bias in LLMs for Healthcare](https://mlsanity.com/paper/2502.13319/) (2025)
- [Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs](https://mlsanity.com/paper/2505.15524/) (2025)
- [From Promising Capability to Pervasive Bias: Assessing Large Language Models for Emergency Department Triage](https://mlsanity.com/paper/2504.16273/) (2025)
- [The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities](https://mlsanity.com/paper/2508.05525/) (2025)
- [LLMs Do Not See Age: Assessing Demographic Bias in Automated Systematic Review Synthesis](https://mlsanity.com/paper/2511.06000/) (2025)
- [Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcare](https://mlsanity.com/paper/2410.18460/) (2024)
- [No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Models](https://mlsanity.com/paper/2503.11985/) (2025)
- [Entry-level guide to the use of large language models for medical research](https://mlsanity.com/paper/2410.18856/) (2024)
- [MEDEQUALQA: Evaluating Biases in LLMs with Counterfactual Reasoning](https://mlsanity.com/paper/2510.12818/) (2025)
- [CT-Bench: A Benchmark for Multimodal Lesion Understanding in Computed Tomography](https://mlsanity.com/paper/2602.14879/) (2026)
- [Whose Journey Matters? Investigating Identity Biases in Large Language Models (LLMs) for Travel Planning Assistance](https://mlsanity.com/paper/2410.17333/) (2024)
- [On the Risk of Misleading Reports: Diagnosing Textual Biases in Multimodal Clinical AI](https://mlsanity.com/paper/2508.00171/) (2025)
- [Can SAEs reveal and mitigate racial biases of LLMs in healthcare?](https://mlsanity.com/paper/2511.00177/) (2025)
- [Dropouts in Confidence: Moral Uncertainty in Human-LLM Alignment](https://mlsanity.com/paper/2511.13290/) (2025)
- [OrthoInsight: Rib Fracture Diagnosis and Report Generation Based on Multi-Modal Large Models](https://mlsanity.com/paper/2507.13993/) (2025)
- [Toward Revealing Nuanced Biases in Medical LLMs](https://mlsanity.com/paper/2507.21176/) (2025)
- [Towards Experience-Centered AI: A Framework for Integrating Lived Experience in Design and Development](https://mlsanity.com/paper/2508.06849/) (2025)
- [HIVMedQA: Benchmarking large language models for HIV medical decision support](https://mlsanity.com/paper/2507.18143/) (2025)
- [A Systematic Review on the Generative AI Applications in Human Medical Genomics](https://mlsanity.com/paper/2508.20275/) (2025)
- [Strategic Innovation Management in the Age of Large Language Models Market Intelligence, Adaptive R&D, and Ethical Governance](https://mlsanity.com/paper/2511.14709/) (2025)
- [Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Once](https://mlsanity.com/paper/2604.01504/) (2026)
- [First, Do No Harm (With LLMs): Mitigating Racial Bias via Agentic Workflows](https://mlsanity.com/paper/2604.18038/) (2026)
- [Trustworthy AI in Digital Health: A Comprehensive Review of Robustness and Explainability](https://mlsanity.com/paper/2608.02238/) (2026)

## Abstract (arXiv)

Large language models like GPT-3.5-turbo and GPT-4 hold promise for healthcare professionals, but they may inadvertently inherit biases during their training, potentially affecting their utility in medical applications. Despite few attempts in the past, the precise impact and extent of these biases remain uncertain. Through both qualitative and quantitative analyses, we find that these models tend to project higher costs and longer hospitalizations for White populations and exhibit optimistic views in challenging medical scenarios with much higher survival rates. These biases, which mirror real-world healthcare disparities, are evident in the generation of patient backgrounds, the association of specific diseases with certain races, and disparities in treatment recommendations, etc. Our findings underscore the critical need for future research to address and mitigate biases in language models, especially in critical healthcare applications, to ensure fair and accurate outcomes for all patients.