# Annotating and Modeling Empathy in Spoken Conversations

arXiv: https://arxiv.org/abs/1705.04839 · HTML: https://mlsanity.com/paper/1705.04839/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2017.

## Built on by (most cited first, 10 of 10)

- [TransESC: Smoothing Emotional Support Conversation via Turn-Level State Transition](https://mlsanity.com/paper/2305.03296/) (2023)
- [EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction](https://mlsanity.com/paper/2102.00272/) (2021)
- [Don't Lose Yourself! Empathetic Response Generation via Explicit Self-Other Awareness](https://mlsanity.com/paper/2210.03884/) (2022)
- [MEDIC: A Multimodal Empathy Dataset in Counseling](https://mlsanity.com/paper/2305.02842/) (2023)
- [Few-Shot Emotion Recognition in Conversation with Sequential Prototypical Networks](https://mlsanity.com/paper/2109.09366/) (2021)
- [Supporting Cognitive and Emotional Empathic Writing of Students](https://mlsanity.com/paper/2105.14815/) (2021)
- [PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction](https://mlsanity.com/paper/2103.03296/) (2021)
- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [An Automated Quality Evaluation Framework of Psychotherapy Conversations with Local Quality Estimates](https://mlsanity.com/paper/2106.07922/) (2021)
- [Bridging Cognition and Emotion: Empathy-Driven Multimodal Misinformation Detection](https://mlsanity.com/paper/2504.17332/) (2025)

## Abstract (arXiv)

Empathy, as defined in behavioral sciences, expresses the ability of human beings to recognize, understand and react to emotions, attitudes and beliefs of others. The lack of an operational definition of empathy makes it difficult to measure it. In this paper, we address two related problems in automatic affective behavior analysis: the design of the annotation protocol and the automatic recognition of empathy from spoken conversations. We propose and evaluate an annotation scheme for empathy inspired by the modal model of emotions. The annotation scheme was evaluated on a corpus of real-life, dyadic spoken conversations. In the context of behavioral analysis, we designed an automatic segmentation and classification system for empathy. Given the different speech and language levels of representation where empathy may be communicated, we investigated features derived from the lexical and acoustic spaces. The feature development process was designed to support both the fusion and automatic selection of relevant features from high dimensional space. The automatic classification system was evaluated on call center conversations where it showed significantly better performance than the baseline.