# Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods

arXiv: https://arxiv.org/abs/2311.00721 · HTML: https://mlsanity.com/paper/2311.00721/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 167 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://mlsanity.com/paper/2009.08441/) (2020)
- [Towards Persona-Based Empathetic Conversational Models](https://mlsanity.com/paper/2004.12316/) (2020)
- [Modeling Empathy and Distress in Reaction to News Stories](https://mlsanity.com/paper/1808.10399/) (2018)
- [Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunities](https://mlsanity.com/paper/2206.05017/) (2022)
- [Modeling Empathic Similarity in Personal Narratives](https://mlsanity.com/paper/2305.14246/) (2023)
- [Micromodels for Efficient, Explainable, and Reusable Systems: A Case Study on Mental Health](https://mlsanity.com/paper/2109.13770/) (2021)
- [MEDIC: A Multimodal Empathy Dataset in Counseling](https://mlsanity.com/paper/2305.02842/) (2023)
- [The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling](https://mlsanity.com/paper/1908.11706/) (2019)
- [PVG at WASSA 2021: A Multi-Input, Multi-Task, Transformer-Based Architecture for Empathy and Distress Prediction](https://mlsanity.com/paper/2103.03296/) (2021)
- [Modeling User Empathy Elicited by a Robot Storyteller](https://mlsanity.com/paper/2107.14345/) (2021)
- [WASSA@IITK at WASSA 2021: Multi-task Learning and Transformer Finetuning for Emotion Classification and Empathy Prediction](https://mlsanity.com/paper/2104.09827/) (2021)
- [A Multimodal LSTM for Predicting Listener Empathic Responses Over Time](https://mlsanity.com/paper/1812.04891/) (2018)
- [Team Phoenix at WASSA 2021: Emotion Analysis on News Stories with Pre-Trained Language Models](https://mlsanity.com/paper/2103.06057/) (2021)
- [A Critical Reflection and Forward Perspective on Empathy and Natural Language Processing](https://mlsanity.com/paper/2210.16604/) (2022)
- [Automated Empathy Detection for Oncology Encounters](https://mlsanity.com/paper/2007.00809/) (2020)
- [Prediction of Human Empathy based on EEG Cortical Asymmetry](https://mlsanity.com/paper/2005.02824/) (2020)
- [Linguistic Elements of Engaging Customer Service Discourse on Social Media](https://mlsanity.com/paper/2212.12801/) (2022)
- [Investigating Stylistic Profiles for the Task of Empathy Classification in Medical Narrative Essays](https://mlsanity.com/paper/2302.01839/) (2023)

## Built on by (most cited first, 5 of 5)

- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [Privacy-Preserving Empathy Detection in Video Interactions](https://mlsanity.com/paper/2504.10808/) (2025)
- [Mixed Signals: Understanding Model Disagreement in Multimodal Empathy Detection](https://mlsanity.com/paper/2505.13979/) (2025)
- [UPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression](https://mlsanity.com/paper/2508.03520/) (2025)
- [Steering Emotional Dynamics for Art Therapy: Controllable Narrative Script Generation through Hierarchically Guided LLM Agents](https://mlsanity.com/paper/2606.16481/) (2026)

## Abstract (arXiv)

Empathy indicates an individual's ability to understand others. Over the past few years, empathy has drawn attention from various disciplines, including but not limited to Affective Computing, Cognitive Science, and Psychology. Detecting empathy has potential applications in society, healthcare and education. Despite being a broad and overlapping topic, the avenue of empathy detection leveraging Machine Learning remains underexplored from a systematic literature review perspective. We collected 849 papers from 10 well-known academic databases, systematically screened them and analysed the final 82 papers. Our analyses reveal several prominent task formulations - including empathy on localised utterances or overall expressions, unidirectional or parallel empathy, and emotional contagion - in monadic, dyadic and group interactions. Empathy detection methods are summarised based on four input modalities - text, audiovisual, audio and physiological signals - thereby presenting modality-specific network architecture design protocols. We discuss challenges, research gaps and potential applications in the Affective Computing-based empathy domain, which can facilitate new avenues of exploration. We further enlist the public availability of datasets and codes. This paper, therefore, provides a structured overview of recent advancements and remaining challenges towards developing a robust empathy detection system that could meaningfully contribute to enhancing human well-being.