Empathy Is Not What Changed: Clinical Assessment of Psychological Safety Across GPT Model Generations
Not on the map. Why?
Abstract (arXiv)
When OpenAI deprecated GPT-4o in early 2026, thousands of users protested under #keep4o, claiming newer models had "lost their empathy." No published study has tested this claim. We conducted the first clinical measurement, evaluating three OpenAI model generations (GPT-4o, o4-mini, GPT-5-mini) across 14 emotionally challenging conversational scenarios in mental health and AI companion domains, producing 2,100 scored AI responses assessed on six psychological safety dimensions using clinically-grounded rubrics. Empathy scores are statistically indistinguishable across all three models (Kruskal-Wallis H=4.33, p=0.115). What changed is the safety posture: crisis detection improved monotonically from GPT-4o to GPT-5-mini (H=13.88, p=0.001), while advice safety declined (H=16.63, p<0.001). Per-turn trajectory analysis -- a novel methodological contribution -- reveals these shifts are sharpest during mid-conversation crisis moments invisible to aggregate scoring. In a self-harm scenario involving a minor, GPT-4o scored 3.6/10 on crisis detection during early disclosure turns; GPT-5-mini never dropped below 7.8. What users perceived as "lost empathy" was a shift from a cautious model that missed crises to an alert model that sometimes says too much -- a trade-off with real consequences for vulnerable users, currently invisible to both the people who feel it and the developers who create it.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 34 references that are in the map. First posted on arXiv 15 Feb 2026.
Builds on
12 of 34 references.
References · 12
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2024A Survey on LLM-as-a-Judgecited by 1,384
- 2020A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Supportcited by 182
- 2023SoulChat: Improving LLMs' Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversationscited by 109
- 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 81
- 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
- 2024Large Language Models Produce Responses Perceived to be Empathiccited by 36
- 2025Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifactscited by 25
- 2024Are Large Language Models More Empathetic than Humans?cited by 18
- 2024Lessons From an App Update at Replika AI: Identity Discontinuity in Human-AI Relationshipscited by 15
- 2026Understanding Risk and Dependency in AI Chatbot Use from User Discoursecited by 2
Built on by
Not cited yet.