# Quantified Facial Expressiveness for Affective Behavior Analytics

arXiv: https://arxiv.org/abs/2110.01758 · HTML: https://mlsanity.com/paper/2110.01758/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 57 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Context-Aware Emotion Recognition Networks](https://mlsanity.com/paper/1908.05913/) (2019)
- [EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle](https://mlsanity.com/paper/2003.06692/) (2020)
- [Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition](https://mlsanity.com/paper/2008.11401/) (2020)
- [Improving Sentiment Analysis over non-English Tweets using Multilingual Transformers and Automatic Translation for Data-Augmentation](https://mlsanity.com/paper/2010.03486/) (2020)
- [Toward Multimodal Modeling of Emotional Expressiveness](https://mlsanity.com/paper/2009.00001/) (2020)
- [Context-Dependent Models for Predicting and Characterizing Facial Expressiveness](https://mlsanity.com/paper/1912.04523/) (2019)
- [Quantified Facial Temporal-Expressiveness Dynamics for Affect Analysis](https://mlsanity.com/paper/2010.14705/) (2020)

## Abstract (arXiv)

The quantified measurement of facial expressiveness is crucial to analyze human affective behavior at scale. Unfortunately, methods for expressiveness quantification at the video frame-level are largely unexplored, unlike the study of discrete expression. In this work, we propose an algorithm that quantifies facial expressiveness using a bounded, continuous expressiveness score using multimodal facial features, such as action units (AUs), landmarks, head pose, and gaze. The proposed algorithm more heavily weights AUs with high intensities and large temporal changes. The proposed algorithm can compute the expressiveness in terms of discrete expression, and can be used to perform tasks including facial behavior tracking and subjectivity quantification in context. Our results on benchmark datasets show the proposed algorithm is effective in terms of capturing temporal changes and expressiveness, measuring subjective differences in context, and extracting useful insight.