# Implicit Media Tagging and Affect Prediction from video of spontaneous facial expressions, recorded with depth camera

arXiv: https://arxiv.org/abs/1701.05248 · HTML: https://mlsanity.com/paper/1701.05248/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2017.

## Built on by (most cited first, 2 of 2)

- [Multimodal Feature Fusion for Video Advertisements Tagging Via Stacking Ensemble](https://mlsanity.com/paper/2108.00679/) (2021)
- [CAST-Phys: Contactless Affective States Through Physiological signals Database](https://mlsanity.com/paper/2507.06080/) (2025)

## Abstract (arXiv)

We present a method that automatically evaluates emotional response from spontaneous facial activity recorded by a depth camera. The automatic evaluation of emotional response, or affect, is a fascinating challenge with many applications, including human-computer interaction, media tagging and human affect prediction. Our approach in addressing this problem is based on the inferred activity of facial muscles over time, as captured by a depth camera recording an individual's facial activity. Our contribution is two-fold: First, we constructed a database of publicly available short video clips, which elicit a strong emotional response in a consistent manner across different individuals. Each video was tagged by its characteristic emotional response along 4 scales: Valence, Arousal, Likability and Rewatch (the desire to watch again). The second contribution is a two-step prediction method, based on learning, which was trained and tested using this database of tagged video clips. Our method was able to successfully predict the aforementioned 4 dimensional representation of affect, as well as to identify the period of strongest emotional response in the viewing recordings, in a method that is blind to the video clip being watch, revealing a significantly high agreement between the recordings of independent viewers.