# Modeling User Empathy Elicited by a Robot Storyteller

arXiv: https://arxiv.org/abs/2107.14345 · HTML: https://mlsanity.com/paper/2107.14345/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 50 references that are in the map.
- First posted on arXiv 29 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning](https://mlsanity.com/paper/1811.12808/) (2018)
- [The OMG-Empathy Dataset: Evaluating the Impact of Affective Behavior in Storytelling](https://mlsanity.com/paper/1908.11706/) (2019)

## Built on by (most cited first, 7 of 7)

- [Multimodal Gender Fairness in Depression Prediction: Insights on Data from the USA & China](https://mlsanity.com/paper/2408.04026/) (2024)
- [Empathy Detection from Text, Audiovisual, Audio or Physiological Signals: A Systematic Review of Task Formulations and Machine Learning Methods](https://mlsanity.com/paper/2311.00721/) (2023)
- [Artificial Empathy Classification: A Survey of Deep Learning Techniques, Datasets, and Evaluation Scales](https://mlsanity.com/paper/2310.00010/) (2023)
- [Turn-Level Empathy Prediction Using Psychological Indicators](https://mlsanity.com/paper/2407.08607/) (2024)
- [Privacy-Preserving Empathy Detection in Video Interactions](https://mlsanity.com/paper/2504.10808/) (2025)
- [Pavlok-Nudge: A Feedback Mechanism for Atomic Behaviour Modification with Snoring Usecase](https://mlsanity.com/paper/2305.06110/) (2023)
- [UpStory: the Uppsala Storytelling dataset](https://mlsanity.com/paper/2407.04352/) (2024)

## Abstract (arXiv)

Virtual and robotic agents capable of perceiving human empathy have the potential to participate in engaging and meaningful human-machine interactions that support human well-being. Prior research in computational empathy has focused on designing empathic agents that use verbal and nonverbal behaviors to simulate empathy and attempt to elicit empathic responses from humans. The challenge of developing agents with the ability to automatically perceive elicited empathy in humans remains largely unexplored. Our paper presents the first approach to modeling user empathy elicited during interactions with a robotic agent. We collected a new dataset from the novel interaction context of participants listening to a robot storyteller (46 participants, 6.9 hours of video). After each storytelling interaction, participants answered a questionnaire that assessed their level of elicited empathy during the interaction with the robot. We conducted experiments with 8 classical machine learning models and 2 deep learning models (long short-term memory networks and temporal convolutional networks) to detect empathy by leveraging patterns in participants' visual behaviors while they were listening to the robot storyteller. Our highest-performing approach, based on XGBoost, achieved an accuracy of 69% and AUC of 72% when detecting empathy in videos. We contribute insights regarding modeling approaches and visual features for automated empathy detection. Our research informs and motivates future development of empathy perception models that can be leveraged by virtual and robotic agents during human-machine interactions.