# The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling

arXiv: https://arxiv.org/abs/1908.11706 · HTML: https://mlsanity.com/paper/1908.11706/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2019.

## Built on by (most cited first, 8 of 8)

- [MEDIC: A Multimodal Empathy Dataset in Counseling](https://mlsanity.com/paper/2305.02842/) (2023)
- [EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences](https://mlsanity.com/paper/2405.15708/) (2024)
- [Modeling User Empathy Elicited by a Robot Storyteller](https://mlsanity.com/paper/2107.14345/) (2021)
- [Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods](https://mlsanity.com/paper/2311.00721/) (2023)
- [Artificial Empathy Classification: A Survey of Deep Learning Techniques, Datasets, and Evaluation Scales](https://mlsanity.com/paper/2310.00010/) (2023)
- [Labels Generated by Large Language Models Help Measure People's Empathy in Vitro](https://mlsanity.com/paper/2501.00691/) (2025)
- [Privacy-Preserving Empathy Detection in Video Interactions](https://mlsanity.com/paper/2504.10808/) (2025)
- [Mixed Signals: Understanding Model Disagreement in Multimodal Empathy Detection](https://mlsanity.com/paper/2505.13979/) (2025)

## Abstract (arXiv)

Processing human affective behavior is important for developing intelligent agents that interact with humans in complex interaction scenarios. A large number of current approaches that address this problem focus on classifying emotion expressions by grouping them into known categories. Such strategies neglect, among other aspects, the impact of the affective responses from an individual on their interaction partner thus ignoring how people empathize towards each other. This is also reflected in the datasets used to train models for affective processing tasks. Most of the recent datasets, in particular, the ones which capture natural interactions ("in-the-wild" datasets), are designed, collected, and annotated based on the recognition of displayed affective reactions, ignoring how these displayed or expressed emotions are perceived. In this paper, we propose a novel dataset composed of dyadic interactions designed, collected and annotated with a focus on measuring the affective impact that eight different stories have on the listener. Each video of the dataset contains around 5 minutes of interaction where a speaker tells a story to a listener. After each interaction, the listener annotated, using a valence scale, how the story impacted their affective state, reflecting how they empathized with the speaker as well as the story. We also propose different evaluation protocols and a baseline that encourages participation in the advancement of the field of artificial empathy and emotion contagion.