# Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset

arXiv: https://arxiv.org/abs/1912.05008 · HTML: https://mlsanity.com/paper/1912.05008/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2019.

## Built on by (most cited first, 20 of 20)

- [Emotion Intensity and its Control for Emotional Voice Conversion](https://mlsanity.com/paper/2201.03967/) (2022)
- [Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality](https://mlsanity.com/paper/2103.06541/) (2021)
- [Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition](https://mlsanity.com/paper/2103.13372/) (2021)
- [Large Language Models are Capable of Offering Cognitive Reappraisal, if Guided](https://mlsanity.com/paper/2404.01288/) (2024)
- [VEATIC: Video-based Emotion and Affect Tracking in Context Dataset](https://mlsanity.com/paper/2309.06745/) (2023)
- [Evaluating Subjective Cognitive Appraisals of Emotions from Large Language Models](https://mlsanity.com/paper/2310.14389/) (2023)
- [Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognition](https://mlsanity.com/paper/2108.00194/) (2021)
- [EmpathicStories++: A Multimodal Dataset for Empathy towards Personal Experiences](https://mlsanity.com/paper/2405.15708/) (2024)
- [MM-AU:Towards Multimodal Understanding of Advertisement Videos](https://mlsanity.com/paper/2308.14052/) (2023)
- [Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses](https://mlsanity.com/paper/2202.10453/) (2022)
- [How you feelin'? Learning Emotions and Mental States in Movie Scenes](https://mlsanity.com/paper/2304.05634/) (2023)
- [An Ethical Framework for Guiding the Development of Affectively-Aware Artificial Intelligence](https://mlsanity.com/paper/2107.13734/) (2021)
- [Modeling Emotional Trajectories in Written Stories Utilizing Transformers and Weakly-Supervised Learning](https://mlsanity.com/paper/2406.02251/) (2024)
- [Does BERT look at sentiment lexicon?](https://mlsanity.com/paper/2111.10100/) (2021)
- [Automatic Emotion Modelling in Written Stories](https://mlsanity.com/paper/2212.11382/) (2022)
- [Whats New? Identifying the Unfolding of New Events in Narratives](https://mlsanity.com/paper/2302.07748/) (2023)
- [Intuitions of Compromise: Utilitarianism vs. Contractualism](https://mlsanity.com/paper/2410.05496/) (2024)
- [Artificial Intelligence for Emotion-Semantic Trending and People Emotion Detection During COVID-19 Social Isolation](https://mlsanity.com/paper/2101.06484/) (2021)
- [Intelligent interactive technologies for mental health and well-being](https://mlsanity.com/paper/2105.05306/) (2021)
- [Understanding Emotion Valence is a Joint Deep Learning Task](https://mlsanity.com/paper/2305.17422/) (2023)

## Abstract (arXiv)

Human emotions unfold over time, and more affective computing research has to prioritize capturing this crucial component of real-world affect. Modeling dynamic emotional stimuli requires solving the twin challenges of time-series modeling and of collecting high-quality time-series datasets. We begin by assessing the state-of-the-art in time-series emotion recognition, and we review contemporary time-series approaches in affective computing, including discriminative and generative models. We then introduce the first version of the Stanford Emotional Narratives Dataset (SENDv1): a set of rich, multimodal videos of self-paced, unscripted emotional narratives, annotated for emotional valence over time. The complex narratives and naturalistic expressions in this dataset provide a challenging test for contemporary time-series emotion recognition models. We demonstrate several baseline and state-of-the-art modeling approaches on the SEND, including a Long Short-Term Memory model and a multimodal Variational Recurrent Neural Network, which perform comparably to the human-benchmark. We end by discussing the implications for future research in time-series affective computing.