mlsanityData to 14 Oct 2026

Elucidating Mechanisms of Demographic Bias in LLMs for Healthcare

arXiv 2502.13319 ↗First posted 18 Feb 2025cs.CLcited by 8

On the map: Mechanistic interpretability → Probing deception, bias and beliefs

Abstract (arXiv)

We know from prior work that LLMs encode social biases, and that this manifests in clinical tasks. In this work we adopt tools from mechanistic interpretability to unveil sociodemographic representations and biases within LLMs in the context of healthcare. Specifically, we ask: Can we identify activations within LLMs that encode sociodemographic information (e.g., gender, race)? We find that gender information is highly localized in MLP layers and can be reliably manipulated at inference time via patching. Such interventions can surgically alter generated clinical vignettes for specific conditions, and also influence downstream clinical predictions which correlate with gender, e.g., patient risk of depression. We find that representation of patient race is somewhat more distributed, but can also be intervened upon, to a degree. To our knowledge, this is the first application of mechanistic interpretability methods to LLMs for healthcare.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. Builds on 26 of its 50 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

26 of 50 references.

References · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  6. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  8. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  11. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  12. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
14 more
  1. 2024How to use and interpret activation patchingcited by 214
  2. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  3. 2020Towards Debiasing Sentence Representationscited by 171
  4. 2020Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddingscited by 79
  5. 2023Evaluating the Zero-shot Robustness of Instruction-tuned Language Modelscited by 66
  6. 2023Summarizing, Simplifying, and Synthesizing Medical Evidence Using GPT-3 (with Varying Success)cited by 40
  7. 2024Unmasking and Quantifying Racial Bias of Large Language Models in Medical Report Generationcited by 31
  8. 2023Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare deliverycited by 30
  9. 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive studycited by 27
  10. 2024Open (Clinical) LLMs are Sensitive to Instruction Phrasingscited by 24
  11. 2023Identifying and Adapting Transformer-Components Responsible for Gender Bias in an English Language Modelcited by 22
  12. 2022Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendationscited by 20
  13. 2024Token Erasure as a Footprint of Implicit Vocabulary Items in LLMscited by 18
  14. 2025Understanding and Mitigating Gender Bias in LLMs via Interpretable Neuron Editingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026Interpretability Can Be Actionablecited by 7
  2. 2025Can SAEs reveal and mitigate racial biases of LLMs in healthcare?cited by 2
  3. 2026Tracing the Latent Threads: A Mechanistic Study of How LLMs Represent and Operationalize Race and Ethnicity Cuescited by 1
  4. 2026A Systematic Evaluation of Large Language Models for PTSD Severity Estimation: The Role of Contextual Knowledge and Modeling Strategiescited by 0
  5. 2026Tracing Pharmacological Knowledge In Large Language Modelscited by 0
  6. 2026Inference Time Causal Probing in LLMscited by 0
  7. 2026Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Groundingcited by 0
  8. 2026Xeno-Interpretability: Investigating the Alien Minds of LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Evaluating the Presence of Sex Bias in Clinical Reasoning by Large Language Models
  2. 2026First, Do No Harm (With LLMs): Mitigating Racial Bias via Agentic Workflows
  3. 2025Evaluating the Bias in LLMs for Surveying Opinion and Decision Making in Healthcare
  4. 2024Mitigating the Risk of Health Inequity Exacerbated by Large Language Models
  5. 2025Surgeons Are Indian Males and Speech Therapists Are White Females: Auditing Biases in Vision-Language Models for Healthcare Professionals
  6. 2024How far can bias go? Tracing bias from pretraining data to alignment
  7. 2025GenderBench: Evaluation Suite for Gender Biases in LLMs
  8. 2025Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective