Can "AI" Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMs
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) are increasingly deployed in healthcare, yet their communicative alignment with clinical standards remains insufficiently quantified. We conduct a multidimensional evaluation of general-purpose and domain-specialized LLMs across structured medical explanations and real-world physician-patient interactions, analyzing semantic fidelity, readability, and affective resonance. Baseline models amplify affective polarity relative to physicians (Very Negative: 43.14-45.10% vs. 37.25%) and, in larger architectures such as GPT-5 and Claude, produce substantially higher linguistic complexity (FKGL up to 16.91-17.60 vs. 11.47-12.50 in physician-authored responses). Empathy-oriented prompting reduces extreme negativity and lowers grade-level complexity (up to -6.87 FKGL points for GPT-5) but does not significantly increase semantic fidelity. Collaborative rewriting yields the strongest overall alignment. Rephrase configurations achieve the highest semantic similarity to physician answers (up to mean = 0.93) while consistently improving readability and reducing affective extremity. Dual stakeholder evaluation shows that no model surpasses physicians on epistemic criteria, whereas patients consistently prefer rewritten variants for clarity and emotional tone. These findings suggest that LLMs function most effectively as collaborative communication enhancers rather than replacements for clinical expertise.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 51 references that are in the map. First posted on arXiv 22 Apr 2026.
Builds on
13 of 51 references.
References · 13
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024Mixtral of Expertscited by 1,823
- 2023Large Language Modelscited by 780
- 2019A Question-Entailment Approach to Question Answeringcited by 134
- 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
- 2024A Framework for Human Evaluation of Large Language Models in Healthcare Derived from Literature Reviewcited by 66
- 2024What are human values, and how do we align AI to them?cited by 32
- 2025A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domainscited by 28
- 2024Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactionscited by 13
- 2023Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstractscited by 8
- 2024An Empirical Evaluation of Large Language Models on Consumer Health Questionscited by 6
- 2025Trustworthy AI for Medicine: Continuous Hallucination Detection and Elimination with CHECKcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Empathy by Design: Aligning Large Language Models for Healthcare Dialogue
- 2025RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systems
- 2025The Dialogue That Heals: A Comprehensive Evaluation of Doctor Agents' Inquiry Capability
- 2026"Do I Trust the AI?" Towards Trustworthy AI-Assisted Diagnosis: Understanding User Perception in LLM-Supported Reasoning
- 2025Enabling Doctor-Centric Medical AI with LLMs through Workflow-Aligned Tasks and Benchmarks
- 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
- 2025Reasoning Is Not All You Need: Examining LLMs for Multi-Turn Mental Health Conversations
- 2026How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting