# Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics

arXiv: https://arxiv.org/abs/2302.09817 · HTML: https://mlsanity.com/paper/2302.09817/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 81 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 9 of 9)

- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [Investigating Audio, Visual, and Text Fusion Methods for End-to-End Automatic Personality Prediction](https://mlsanity.com/paper/1805.00705/) (2018)
- [Automated Analysis and Prediction of Job Interview Performance](https://mlsanity.com/paper/1504.03425/) (2015)
- [First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis](https://mlsanity.com/paper/1804.08046/) (2018)
- [Multichannel Attention Network for Analyzing Visual Behavior in Public Speaking](https://mlsanity.com/paper/1707.06830/) (2017)
- [Head Matters: Explainable Human-centered Trait Prediction from Head Motion Dynamics](https://mlsanity.com/paper/2112.08068/) (2021)
- [Learning Spontaneity to Improve Emotion Recognition In Speech](https://mlsanity.com/paper/1712.04753/) (2017)
- [Leveraging Multimodal Behavioral Analytics for Automated Job Interview Performance Assessment and Feedback](https://mlsanity.com/paper/2006.07909/) (2020)
- [Characterizing Hirability via Personality and Behavior](https://mlsanity.com/paper/2006.12041/) (2020)

## Built on by (most cited first, 2 of 2)

- [MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings](https://mlsanity.com/paper/2309.10765/) (2023)
- [GAViD: A Large-Scale Multimodal Dataset for Context-Aware Group Affect Recognition from Videos](https://mlsanity.com/paper/2604.16214/) (2026)

## Abstract (arXiv)

We explore the efficacy of multimodal behavioral cues for explainable prediction of personality and interview-specific traits. We utilize elementary head-motion units named kinemes, atomic facial movements termed action units and speech features to estimate these human-centered traits. Empirical results confirm that kinemes and action units enable discovery of multiple trait-specific behaviors while also enabling explainability in support of the predictions. For fusing cues, we explore decision and feature-level fusion, and an additive attention-based fusion strategy which quantifies the relative importance of the three modalities for trait prediction. Examining various long-short term memory (LSTM) architectures for classification and regression on the MIT Interview and First Impressions Candidate Screening (FICS) datasets, we note that: (1) Multimodal approaches outperform unimodal counterparts; (2) Efficient trait predictions and plausible explanations are achieved with both unimodal and multimodal approaches, and (3) Following the thin-slice approach, effective trait prediction is achieved even from two-second behavioral snippets.