# FairMedQA: Benchmarking Bias in Large Language Models for Medical Question Answering

arXiv: https://arxiv.org/abs/2505.19562 · HTML: https://mlsanity.com/paper/2505.19562/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Capabilities of Gemini Models in Medicine](https://mlsanity.com/paper/2404.18416/) (2024)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Bias in Machine Learning Software: Why? How? What to do?](https://mlsanity.com/paper/2105.12195/) (2021)
- [FairCLIP: Harnessing Fairness in Vision-Language Learning](https://mlsanity.com/paper/2403.19949/) (2024)
- [Fairness Testing: A Comprehensive Survey and Analysis of Trends](https://mlsanity.com/paper/2207.10223/) (2022)
- [A Survey On Universal Adversarial Attack](https://mlsanity.com/paper/2103.01498/) (2021)
- [Addressing cognitive bias in medical language models](https://mlsanity.com/paper/2402.08113/) (2024)
- [MedFuzz: Exploring the Robustness of Large Language Models in Medical Question Answering](https://mlsanity.com/paper/2406.06573/) (2024)
- [MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications](https://mlsanity.com/paper/2409.07314/) (2024)
- [Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias](https://mlsanity.com/paper/2405.05506/) (2024)
- [FMBench: Benchmarking Fairness in Multimodal Large Language Models on Medical Tasks](https://mlsanity.com/paper/2410.01089/) (2024)
- [Causality-Aided Trade-off Analysis for Machine Learning Fairness](https://mlsanity.com/paper/2305.13057/) (2023)
- [Enabling Scalable Evaluation of Bias Patterns in Medical LLMs](https://mlsanity.com/paper/2410.14763/) (2024)

## Built on by (most cited first, 6 of 6)

- [Can Agents Fix Agent Issues?](https://mlsanity.com/paper/2505.20749/) (2025)
- [AgentFairBench: Do LLM Agents Discriminate When They Act?](https://mlsanity.com/paper/2606.16723/) (2026)
- [Rethinking Evidence Hierarchies in Medical Language Benchmarks: A Critical Evaluation of HealthBench](https://mlsanity.com/paper/2508.00081/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting](https://mlsanity.com/paper/2512.19735/) (2025)
- [AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems](https://mlsanity.com/paper/2609.37864/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are approaching expert-level performance in medical question answering (QA), demonstrating strong potential to improve public healthcare. However, underlying biases related to sensitive attributes such as sex and race pose life-critical risks. The extent to which such sensitive attributes affect diagnosis remains an open question and requires comprehensive empirical investigation. Additionally, even the latest Counterfactual Patient Variations (CPV) benchmark can hardly distinguish the bias levels of different LLMs. To further explore these dynamics, we propose a new benchmark, FairMedQA, and benchmark 12 representative LLMs. FairMedQA contains 4,806 counterfactual question pairs constructed from 801 clinical vignettes. Our results reveal substantial accuracy disparity ranging from 3 to 19 percentage points across sensitive demographic groups. Notably, FairMedQA exposes biases that are at least 12 percentage points larger than those identified by the latest CPV benchmark, presenting superior benchmarking sensitivity. Our results underscore an urgent need for targeted debiasing techniques and more rigorous, identity-aware validation protocols before LLMs can be safely integrated into practical clinical decision-support systems.