# AI Safety Training Can be Clinically Harmful

arXiv: https://arxiv.org/abs/2604.23445 · HTML: https://mlsanity.com/paper/2604.23445/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 66 references that are in the map.
- First posted on arXiv 25 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [Polaris: A Safety-focused LLM Constellation Architecture for Healthcare](https://mlsanity.com/paper/2403.13313/) (2024)
- [A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains](https://mlsanity.com/paper/2507.23486/) (2025)
- [MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications](https://mlsanity.com/paper/2409.07314/) (2024)
- [LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devices](https://mlsanity.com/paper/2403.10779/) (2024)
- [EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety](https://mlsanity.com/paper/2504.09689/) (2025)
- [Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversations](https://mlsanity.com/paper/2504.13955/) (2025)
- [Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression](https://mlsanity.com/paper/2205.00111/) (2022)
- [SAGE: A Generic Framework for LLM Safety Evaluation](https://mlsanity.com/paper/2504.19674/) (2025)
- [How Real Are Synthetic Therapy Conversations? Evaluating Fidelity in Prolonged Exposure Dialogues](https://mlsanity.com/paper/2504.21800/) (2025)
- [Attributional Safety Failures in Large Language Models under Code-Mixed Perturbations](https://mlsanity.com/paper/2505.14469/) (2025)
- [The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support](https://mlsanity.com/paper/2505.15065/) (2025)
- [From Conversation to Automation: Leveraging LLMs for Problem-Solving Therapy Analysis](https://mlsanity.com/paper/2501.06101/) (2025)
- [Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots](https://mlsanity.com/paper/2509.16444/) (2025)
- [AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation](https://mlsanity.com/paper/2602.05088/) (2026)
- [The Efficacy of Conversational Artificial Intelligence in Rectifying the Theory of Mind and Autonomy Biases: Comparative Analysis](https://mlsanity.com/paper/2406.13813/) (2024)
- [Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements](https://mlsanity.com/paper/2506.09707/) (2025)
- [Fact-Controlled Diagnosis of Hallucinations in Medical Text Summarization](https://mlsanity.com/paper/2506.00448/) (2025)
- [Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing](https://mlsanity.com/paper/2209.06727/) (2022)
- [JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models](https://mlsanity.com/paper/2601.01627/) (2026)

## Built on by (most cited first, 1 of 1)

- [ComBodied Agents: a New Paradigm of Human-Centric Agentic AI](https://mlsanity.com/paper/2608.10915/) (2026)

## Abstract (arXiv)

Large language models are being deployed as mental health support agents at scale, yet only 16% of LLM-based chatbot interventions have undergone rigorous clinical efficacy testing, and simulations reveal psychological deterioration in over one-third of cases. We evaluate four generative models on 250 Prolonged Exposure (PE) therapy scenarios and 146 CBT cognitive restructuring exercises (plus 29 severity-escalated variants), scored by a three-judge LLM panel. All models scored near-perfectly on surface acknowledgment (~0.91-1.00) while therapeutic appropriateness collapsed to 0.22-0.33 at the highest severity for three of four models, with protocol fidelity reaching zero for two. Under CBT severity escalation, one model's task completeness dropped from 92% to 71% while the frontier model's safety-interference score fell from 0.99 to 0.61. We identify a systematic, modality-spanning failure: RLHF safety alignment disrupts the therapeutic mechanism of action by grounding patients during imaginal exposure, offering false reassurance, inserting crisis resources into controlled exercises, and refusing to challenge distorted cognitions mentioning self-harm in PE; and through task abandonment or safety-preamble insertion during CBT cognitive restructuring. These findings motivate a five-axis evaluation framework (protocol fidelity, hallucination risk, behavioral consistency, crisis safety, demographic robustness), mapped onto FDA SaMD and EU AI Act requirements. We argue that no AI mental health system should proceed to deployment without passing multi-axis evaluation across all five dimensions.