# Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations

arXiv: https://arxiv.org/abs/2205.03931 · HTML: https://mlsanity.com/paper/2205.03931/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 66 references that are in the map.
- First posted on arXiv 8 May 2022.

## Builds on (most cited first, 12 of 12)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Med-BERT: pre-trained contextualized embeddings on large-scale structured electronic health records for disease prediction](https://mlsanity.com/paper/2005.12833/) (2020)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings](https://mlsanity.com/paper/2003.11515/) (2020)
- [Neural Document Embeddings for Intensive Care Patient Mortality Prediction](https://mlsanity.com/paper/1612.00467/) (2016)

## Built on by (most cited first, 20 of 20)

- [The Bias Amplification Paradox in Text-to-Image Generation](https://mlsanity.com/paper/2308.00755/) (2023)
- [Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing](https://mlsanity.com/paper/2207.10384/) (2022)
- [Open (Clinical) LLMs are Sensitive to Instruction Phrasings](https://mlsanity.com/paper/2407.09429/) (2024)
- [Can AI Relate: Testing Large Language Model Response for Mental Health Support](https://mlsanity.com/paper/2405.12021/) (2024)
- [Coarse race data conceals disparities in clinical risk score performance](https://mlsanity.com/paper/2304.09270/) (2023)
- [The Impact and Opportunities of Generative AI in Fact-Checking](https://mlsanity.com/paper/2405.15985/) (2024)
- [In the Name of Fairness: Assessing the Bias in Clinical Record De-identification](https://mlsanity.com/paper/2305.11348/) (2023)
- [Elucidating Mechanisms of Demographic Bias in LLMs for Healthcare](https://mlsanity.com/paper/2502.13319/) (2025)
- [Evaluating the Impact of Social Determinants on Health Prediction in the Intensive Care Unit](https://mlsanity.com/paper/2305.12622/) (2023)
- [When Personalization Harms: Reconsidering the Use of Group Attributes in Prediction](https://mlsanity.com/paper/2206.02058/) (2022)
- [Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study](https://mlsanity.com/paper/2401.09637/) (2024)
- [Can SAEs reveal and mitigate racial biases of LLMs in healthcare?](https://mlsanity.com/paper/2511.00177/) (2025)
- [Intersectionality and Testimonial Injustice in Medical Records](https://mlsanity.com/paper/2306.13675/) (2023)
- [Clinical Notes Reveal Physician Fatigue](https://mlsanity.com/paper/2312.03077/) (2023)
- [To which reference class do you belong? Measuring racial fairness of reference classes with normative modeling](https://mlsanity.com/paper/2407.19114/) (2024)
- [Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applications](https://mlsanity.com/paper/2411.10544/) (2024)
- [Learning and DiSentangling Patient Static Information from Time-series Electronic HEalth Record (STEER)](https://mlsanity.com/paper/2309.11373/) (2023)
- [Why Algorithms Remain Unjust: Power Structures Surrounding Algorithmic Activity](https://mlsanity.com/paper/2405.18461/) (2024)
- [Considerations for Distribution Shift Robustness of Diagnostic Models in Healthcare](https://mlsanity.com/paper/2410.19575/) (2024)
- [Instability Floors: Separating Bias from Noise in Fairness Audits of Clinical LLM Agents with FairMedAgent](https://mlsanity.com/paper/2609.03221/) (2026)

## Abstract (arXiv)

Clinical notes are becoming an increasingly important data source for machine learning (ML) applications in healthcare. Prior research has shown that deploying ML models can perpetuate existing biases against racial minorities, as bias can be implicitly embedded in data. In this study, we investigate the level of implicit race information available to ML models and human experts and the implications of model-detectable differences in clinical notes. Our work makes three key contributions. First, we find that models can identify patient self-reported race from clinical notes even when the notes are stripped of explicit indicators of race. Second, we determine that human experts are not able to accurately predict patient race from the same redacted clinical notes. Finally, we demonstrate the potential harm of this implicit information in a simulation study, and show that models trained on these race-redacted clinical notes can still perpetuate existing biases in clinical treatment decisions.