# Recognizing Emotions evoked by Movies using Multitask Learning

arXiv: https://arxiv.org/abs/2107.14529 · HTML: https://mlsanity.com/paper/2107.14529/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 41 references that are in the map.
- First posted on arXiv 30 Jul 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Identifying Stable Patterns over Time for Emotion Recognition from EEG](https://mlsanity.com/paper/1601.02197/) (2016)
- [Multimodal Emotion Recognition Using Deep Canonical Correlation Analysis](https://mlsanity.com/paper/1908.05349/) (2019)
- [Multimodal Deep Models for Predicting Affective Responses Evoked by Movies](https://mlsanity.com/paper/1909.06957/) (2019)
- [Probabilistic Multigraph Modeling for Improving the Quality of Crowdsourced Affective Data](https://mlsanity.com/paper/1701.01096/) (2017)
- [Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1705.02394/) (2017)

## Built on by (most cited first, 1 of 1)

- [Learning Annotation Consensus for Continuous Emotion Recognition](https://mlsanity.com/paper/2505.21196/) (2025)

## Abstract (arXiv)

Understanding the emotional impact of movies has become important for affective movie analysis, ranking, and indexing. Methods for recognizing evoked emotions are usually trained on human annotated data. Concretely, viewers watch video clips and have to manually annotate the emotions they experienced while watching the videos. Then, the common practice is to aggregate the different annotations, by computing average scores or majority voting, and train and test models on these aggregated annotations. With this procedure a single aggregated evoked emotion annotation is obtained per each video. However, emotions experienced while watching a video are subjective: different individuals might experience different emotions. In this paper, we model the emotions evoked by videos in a different manner: instead of modeling the aggregated value we jointly model the emotions experienced by each viewer and the aggregated value using a multi-task learning approach. Concretely, we propose two deep learning architectures: a Single-Task (ST) architecture and a Multi-Task (MT) architecture. Our results show that the MT approach can more accurately model each viewer and the aggregated annotation when compared to methods that are directly trained on the aggregated annotations. Furthermore, our approach outperforms the current state-of-the-art results on the COGNIMUSE benchmark.