# The Pursuit of Empathy: Evaluating Small Language Models for PTSD Dialogue Support

arXiv: https://arxiv.org/abs/2505.15065 · HTML: https://mlsanity.com/paper/2505.15065/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 67 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 22 of 22)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [On Measuring Social Biases in Sentence Encoders](https://mlsanity.com/paper/1903.10561/) (2019)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation](https://mlsanity.com/paper/2105.08316/) (2021)
- [Harnessing the Power of Large Language Models for Empathetic Response Generation: Empirical Investigations and Improvements](https://mlsanity.com/paper/2310.05140/) (2023)
- [Empowering Psychotherapy with Large Language Models: Cognitive Distortion Detection through Diagnosis of Thought Prompting](https://mlsanity.com/paper/2310.07146/) (2023)
- [Chain of Empathy: Enhancing Empathetic Response of Large Language Models Based on Psychotherapy Models](https://mlsanity.com/paper/2311.04915/) (2023)
- [Harnessing Large Language Models' Empathetic Response Generation Capabilities for Online Mental Health Counselling Support](https://mlsanity.com/paper/2310.08017/) (2023)
- [Enhancing Empathetic Response Generation by Augmenting LLMs with Small-scale Empathetic Models](https://mlsanity.com/paper/2402.11801/) (2024)
- [Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversations](https://mlsanity.com/paper/2504.13955/) (2025)
- [Multi-dimensional Evaluation of Empathetic Dialog Responses](https://mlsanity.com/paper/2402.11409/) (2024)
- [Privacy Sensitive Speech Analysis Using Federated Learning to Assess Depression](https://mlsanity.com/paper/2205.00111/) (2022)
- [How Real Are Synthetic Therapy Conversations? Evaluating Fidelity in Prolonged Exposure Dialogues](https://mlsanity.com/paper/2504.21800/) (2025)
- [A Novel Nuanced Conversation Evaluation Framework for Large Language Models in Mental Health](https://mlsanity.com/paper/2403.09705/) (2024)
- [Alexa as an Active Listener: How Backchanneling Can Elicit Self-Disclosure and Promote User Experience](https://mlsanity.com/paper/2204.10191/) (2022)
- [Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy](https://mlsanity.com/paper/2409.00112/) (2024)
- [Enhancing Mental Health Support through Human-AI Collaboration: Toward Secure and Empathetic AI-enabled chatbots](https://mlsanity.com/paper/2410.02783/) (2024)
- [Towards a Multidimensional Evaluation Framework for Empathetic Conversational Systems](https://mlsanity.com/paper/2407.18538/) (2024)

## Built on by (most cited first, 6 of 6)

- [PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models](https://mlsanity.com/paper/2601.10532/) (2026)
- [Fine-Tuning Large Audio-Language Models with LoRA for Precise Temporal Localization of Prolonged Exposure Therapy Elements](https://mlsanity.com/paper/2506.09707/) (2025)
- [Responsible Evaluation of AI for Mental Health](https://mlsanity.com/paper/2602.00065/) (2026)
- [AI Safety Training Can be Clinically Harmful](https://mlsanity.com/paper/2604.23445/) (2026)
- [From Empathy to Personalized Empathy: Adapting Empathetic Strategies to Individual Users](https://mlsanity.com/paper/2606.00728/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)

## Abstract (arXiv)

This paper investigates the capacity of small language models (0.5B-5B parameters) to generate empathetic responses for individuals with PTSD. We introduce Trauma-Informed Dialogue for Empathy (TIDE), a novel dataset comprising 10,000 two-turn conversations across 500 diverse, clinically-grounded PTSD personas (https://huggingface.co/datasets/yenopoya/TIDE). Using frontier model outputs as ground truth, we evaluate eight small LLMs in zero-shot settings and after fine-tuning. Fine-tuning enhances empathetic capabilities, improving cosine similarity and perceived empathy, although gains vary across emotional scenarios and smaller models exhibit a "knowledge transfer ceiling." As expected, Claude Sonnet 3.5 consistently outperforms all models, but surprisingly, the smaller models often approach human-rated empathy levels. Demographic analyses showed that older adults favored responses that validated distress before offering support (p = .004), while graduate-educated users preferred emotionally layered replies in specific scenarios. Gender-based differences were minimal (p > 0.15), suggesting the feasibility of broadly empathetic model designs. This work offers insights into building resource-efficient, emotionally intelligent systems for mental health support.