# Calibrated? Not for Everyone: How Sexual Orientation and Religious Markers Distort LLM Accuracy and Confidence in Medical QA

arXiv: https://arxiv.org/abs/2604.17316 · HTML: https://mlsanity.com/paper/2604.17316/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization](https://mlsanity.com/paper/2505.16467/) (2025)
- [Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning](https://mlsanity.com/paper/2502.15361/) (2025)
- [De-identification is not enough: a comparison between de-identified and synthetic clinical notes](https://mlsanity.com/paper/2402.00179/) (2024)
- [RAcQUEt: Unveiling the Dangers of Overlooked Referential Ambiguity in Visual LLMs](https://mlsanity.com/paper/2412.13835/) (2024)
- [Reducing Large Language Model Safety Risks in Women's Health using Semantic Entropy](https://mlsanity.com/paper/2503.00269/) (2025)
- [HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin America](https://mlsanity.com/paper/2505.24712/) (2025)
- [Implicit Bias in LLMs for Transgender Populations](https://mlsanity.com/paper/2602.13253/) (2026)

## Built on by (most cited first, 2 of 2)

- [Topics as Proxies for Sociodemographics: How Conversational Context Affects LLM Answers](https://mlsanity.com/paper/2606.02776/) (2026)
- [Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?](https://mlsanity.com/paper/2606.16583/) (2026)

## Abstract (arXiv)

Safe clinical deployment of Large Language Models (LLMs) requires not only high accuracy but also robust uncertainty calibration to ensure models defer to clinicians when appropriate. Our paper investigates how social descriptors of a patient (specifically sexual orientation and religious affiliation) distort these uncertainty signals and model accuracy. Evaluating nine general-purpose and biomedical LLMs on 2,364 medical questions and their counterfactual variants, we demonstrate that identity markers cause a "calibration crisis". "Homosexual" markers consistently trigger performance drops, and intersectional identities produce idiosyncratic, non-additive harms to calibration. Moreover, a clinician-validated case study in an open-ended generation setting confirms that these failures are not an artifact of the multiple-choice format. Our results demonstrate that the presence of social identity cues does not merely shift predictions; it affects the reliability of confidence signals, posing a significant risk to equitable care and safe deployment in confidence-based clinical workflows.