# Saliency-guided Emotion Modeling: Predicting Viewer Reactions from Video Stimuli

arXiv: https://arxiv.org/abs/2505.19178 · HTML: https://mlsanity.com/paper/2505.19178/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 25 May 2025.

## Builds on (most cited first, 7 of 7)

- [Deep Visual Attention Prediction](https://mlsanity.com/paper/1705.02544/) (2017)
- [Revisiting Video Saliency: A Large-scale Benchmark and a New Model](https://mlsanity.com/paper/1801.07424/) (2018)
- [TASED-Net: Temporally-Aggregating Spatial Encoder-Decoder Network for Video Saliency Detection](https://mlsanity.com/paper/1908.05786/) (2019)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusing](https://mlsanity.com/paper/1704.03067/) (2017)
- [How is Gaze Influenced by Image Transformations? Dataset and Model](https://mlsanity.com/paper/1905.06803/) (2019)
- [Hierarchical Domain-Adapted Feature Learning for Video Saliency Prediction](https://mlsanity.com/paper/2010.01220/) (2020)

## Abstract (arXiv)

Understanding the emotional impact of videos is crucial for applications in content creation, advertising, and Human-Computer Interaction (HCI). Traditional affective computing methods rely on self-reported emotions, facial expression analysis, and biosensing data, yet they often overlook the role of visual saliency -- the naturally attention-grabbing regions within a video. In this study, we utilize deep learning to introduce a novel saliency-based approach to emotion prediction by extracting two key features: saliency area and number of salient regions. Using the HD2S saliency model and OpenFace facial action unit analysis, we examine the relationship between video saliency and viewer emotions. Our findings reveal three key insights: (1) Videos with multiple salient regions tend to elicit high-valence, low-arousal emotions, (2) Videos with a single dominant salient region are more likely to induce low-valence, high-arousal responses, and (3) Self-reported emotions often misalign with facial expression-based emotion detection, suggesting limitations in subjective reporting. By leveraging saliency-driven insights, this work provides a computationally efficient and interpretable alternative for emotion modeling, with implications for content creation, personalized media experiences, and affective computing research.