# When Large Language Models are Reliable for Judging Empathic Communication

arXiv: https://arxiv.org/abs/2506.10150 · HTML: https://mlsanity.com/paper/2506.10150/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 94 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs](https://mlsanity.com/paper/2502.17424/) (2025)
- [ParlAI: A Dialog Research Software Platform](https://mlsanity.com/paper/1705.06476/) (2017)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [The Generative AI Paradox: "What It Can Create, It May Not Understand"](https://mlsanity.com/paper/2311.00059/) (2023)
- [Wider and Deeper LLM Networks are Fairer LLM Evaluators](https://mlsanity.com/paper/2308.01862/) (2023)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization](https://mlsanity.com/paper/2311.09184/) (2023)
- [Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers](https://mlsanity.com/paper/2504.18412/) (2025)
- [Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support](https://mlsanity.com/paper/2203.15144/) (2022)
- [A Computational Framework for Behavioral Assessment of LLM Therapists](https://mlsanity.com/paper/2401.00820/) (2024)
- [Why human-AI relationships need socioaffective alignment](https://mlsanity.com/paper/2502.02528/) (2025)
- [Large Language Models Produce Responses Perceived to be Empathic](https://mlsanity.com/paper/2403.18148/) (2024)
- [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022)
- [Annotation alignment: Comparing LLM and human annotations of conversational safety](https://mlsanity.com/paper/2406.06369/) (2024)
- [Can AI Relate: Testing Large Language Model Response for Mental Health Support](https://mlsanity.com/paper/2405.12021/) (2024)
- [Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs](https://mlsanity.com/paper/2402.11958/) (2024)
- [HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs](https://mlsanity.com/paper/2405.17633/) (2024)
- [Social Skill Training with Large Language Models](https://mlsanity.com/paper/2404.04204/) (2024)
- [Rankers, Judges, and Assistants: Towards Understanding the Interplay of LLMs in Information Retrieval Evaluation](https://mlsanity.com/paper/2503.19092/) (2025)
- [Scaffolding Empathy: Training Counselors with Simulated Patients and Utterance-level Performance Visualizations](https://mlsanity.com/paper/2502.18673/) (2025)
- [AI on My Shoulder: Supporting Emotional Labor in Front-Office Roles with an LLM-based Empathetic Coworker](https://mlsanity.com/paper/2411.02408/) (2024)
- [Talk, Listen, Connect: How Humans and AI Evaluate Empathy in Responses to Emotionally Charged Narratives](https://mlsanity.com/paper/2409.15550/) (2024)
- [Computational Empathy Counteracts the Negative Effects of Anger on Creative Problem Solving](https://mlsanity.com/paper/2208.07178/) (2022)

## Built on by (most cited first, 21 of 21)

- [Characterizing Delusional Spirals through Human-LLM Chat Logs](https://mlsanity.com/paper/2603.16567/) (2026)
- [HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue](https://mlsanity.com/paper/2601.19922/) (2026)
- [SENSE-7: Taxonomy and Dataset for Measuring User Perceptions of Empathy in Sustained Human-AI Conversations](https://mlsanity.com/paper/2509.16437/) (2025)
- [AI generates well-liked but templatic empathic responses](https://mlsanity.com/paper/2604.08479/) (2026)
- [Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?](https://mlsanity.com/paper/2605.16354/) (2026)
- [Enhancing LLM-Based Data Annotation with Error Decomposition](https://mlsanity.com/paper/2601.11920/) (2026)
- [EFT-CoT: A Multi-Agent Chain-of-Thought Framework for Emotion-Focused Therapy](https://mlsanity.com/paper/2601.17842/) (2026)
- [Discourse Diversity in Multi-Turn Empathic Dialogue](https://mlsanity.com/paper/2604.11742/) (2026)
- [NeuroAI and Beyond: Bridging Between Advances in Neuroscience and Artificial Intelligence](https://mlsanity.com/paper/2604.18637/) (2026)
- [AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models](https://mlsanity.com/paper/2604.20996/) (2026)
- [When Seekers Are Hard to Help: Evaluating Emotional Support Dialogue Systems in Worst-Case Interactions](https://mlsanity.com/paper/2605.28228/) (2026)
- [Lost in Delusion: Examining LLM Safety Under User Delusions and Distress](https://mlsanity.com/paper/2606.00975/) (2026)
- [Empathy Applicability Modeling for General Health Queries](https://mlsanity.com/paper/2601.09696/) (2026)
- [Spontaneous Persuasion: An Audit of Model Persuasiveness in Everyday Conversations](https://mlsanity.com/paper/2604.22109/) (2026)
- [Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthropomorphism, and Maxims](https://mlsanity.com/paper/2606.02493/) (2026)
- [Self-Stigma Is Not a Monolith, but Generic Empathy Is: Persona-Conditioned LLM Support for People Who Use Drugs](https://mlsanity.com/paper/2606.23387/) (2026)
- [Collective cooperation without individual fidelity in LLM agents](https://mlsanity.com/paper/2606.30454/) (2026)
- [Speech Signals Complement LLMs for Predicting Interpersonal Attraction in Speed Dating](https://mlsanity.com/paper/2607.23037/) (2026)
- [Empathy Is Steerable but Multi-Axial: Mechanism Geometry and Persona Effects in LLMs](https://mlsanity.com/paper/2609.15654/) (2026)
- [Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling](https://mlsanity.com/paper/2609.22934/) (2026)
- [How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstats](https://mlsanity.com/paper/2609.35815/) (2026)

## Abstract (arXiv)

Large language models (LLMs) excel at generating empathic responses in text-based conversations. But, how reliably do they judge the nuances of empathic communication? We investigate this question by comparing how experts, crowdworkers, and LLMs annotate empathic communication across four evaluative frameworks drawn from psychology, natural language processing, and communications applied to 200 real-world conversations where one speaker shares a personal problem and the other offers support. Drawing on 3,150 expert annotations, 2,844 crowd annotations, and 3,150 LLM annotations, we assess inter-rater reliability between these three annotator groups. We find that expert agreement is high but varies across the frameworks' sub-components depending on their clarity, complexity, and subjectivity. We show that expert agreement offers a more informative benchmark for contextualizing LLM performance than standard classification metrics. Across all four frameworks, LLMs consistently approach this expert level benchmark and exceed the reliability of crowdworkers. These results demonstrate how LLMs, when validated on specific tasks with appropriate benchmarks, can support transparency and oversight in emotionally sensitive applications including their use as conversational companions.