# First, Do No Harm (With LLMs): Mitigating Racial Bias via Agentic Workflows

arXiv: https://arxiv.org/abs/2604.18038 · HTML: https://mlsanity.com/paper/2604.18038/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 38 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 10 of 10)

- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Large language models in healthcare and medical domain: A review](https://mlsanity.com/paper/2401.06775/) (2023)
- [Measuring Implicit Bias in Explicitly Unbiased Large Language Models](https://mlsanity.com/paper/2402.04105/) (2024)
- [Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generation](https://mlsanity.com/paper/2401.13867/) (2024)
- [A Comprehensive Survey of Bias in LLMs: Current Landscape and Future Directions](https://mlsanity.com/paper/2409.16430/) (2024)
- [LLMs-Healthcare : Current Applications and Challenges of Large Language Models in various Medical Specialties](https://mlsanity.com/paper/2311.12882/) (2023)
- [A Comprehensive Analysis of Large Language Model Outputs: Similarity, Diversity, and Bias](https://mlsanity.com/paper/2505.09056/) (2025)
- [Research-Oriented Human-Centric Evaluation for Foundation Models](https://mlsanity.com/paper/2506.01793/) (2025)
- [An Agentic AI Workflow for Detecting Cognitive Concerns in Real-world Data](https://mlsanity.com/paper/2502.01789/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used in clinical settings, raising concerns about racial bias in both generated medical text and clinical reasoning. Existing studies have identified bias in medical LLMs, but many focus on single models and give less attention to mitigation. This study uses the EU AI Act as a governance lens to evaluate five widely used LLMs across two tasks, namely synthetic patient-case generation and differential diagnosis ranking. Using race-stratified epidemiological distributions in the United States and expert differential diagnosis lists as benchmarks, we apply structured prompt templates and a two-part evaluation design to examine implicit and explicit racial bias. All models deviated from observed racial distributions in the synthetic case generation task, with GPT-4.1 showing the smallest overall deviation. In the differential diagnosis task, DeepSeek V3 produced the strongest overall results across the reported metrics. When embedded in an agentic workflow, DeepSeek V3 showed an improvement of 0.0348 in mean p-value, 0.1166 in median p-value, and 0.0949 in mean difference relative to the standalone model, although improvement was not uniform across every metric. These findings support multi-metric bias evaluation for AI systems used in medical settings and suggest that retrieval-based agentic workflows may reduce some forms of explicit bias in benchmarked diagnostic tasks. Detailed prompt templates, experimental datasets, and code pipelines are available on our GitHub.