# Human-Centered Emotion Recognition in Animated GIFs

arXiv: https://arxiv.org/abs/1904.12201 · HTML: https://mlsanity.com/paper/1904.12201/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2019.

## Built on by (most cited first, 4 of 4)

- [Dual-path Collaborative Generation Network for Emotional Video Captioning](https://mlsanity.com/paper/2408.03006/) (2024)
- [CAST: Character labeling in Animation using Self-supervision by Tracking](https://mlsanity.com/paper/2201.07619/) (2022)
- [Towards Interpretability in Audio and Visual Affective Machine Learning: A Review](https://mlsanity.com/paper/2306.08933/) (2023)
- [EmoDiarize: Speaker Diarization and Emotion Identification from Speech Signals using Convolutional Neural Networks](https://mlsanity.com/paper/2310.12851/) (2023)

## Abstract (arXiv)

As an intuitive way of expression emotion, the animated Graphical Interchange Format (GIF) images have been widely used on social media. Most previous studies on automated GIF emotion recognition fail to effectively utilize GIF's unique properties, and this potentially limits the recognition performance. In this study, we demonstrate the importance of human related information in GIFs and conduct human-centered GIF emotion recognition with a proposed Keypoint Attended Visual Attention Network (KAVAN). The framework consists of a facial attention module and a hierarchical segment temporal module. The facial attention module exploits the strong relationship between GIF contents and human characters, and extracts frame-level visual feature with a focus on human faces. The Hierarchical Segment LSTM (HS-LSTM) module is then proposed to better learn global GIF representations. Our proposed framework outperforms the state-of-the-art on the MIT GIFGIF dataset. Furthermore, the facial attention module provides reliable facial region mask predictions, which improves the model's interpretability.