# Are 3D Face Shapes Expressive Enough for Recognising Continuous Emotions and Action Unit Intensities? arXiv: https://arxiv.org/abs/2207.01113 ยท HTML: https://mlsanity.com/paper/2207.01113/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 77 references that are in the map. - First posted on arXiv 3 Jul 2022. ## Builds on (most cited first, 30 of 30) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016) - [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017) - [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020) - [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019) - [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022) - [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020) - [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020) - [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019) - [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022) - [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018) - [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016) - [AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition](https://mlsanity.com/paper/1907.11510/) (2019) - [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018) - [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019) - [Factorized Higher-Order CNNs with an Application to Spatio-Temporal Emotion Estimation](https://mlsanity.com/paper/1906.06196/) (2019) - [Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Dataset](https://mlsanity.com/paper/1910.01417/) (2019) - [ExpNet: Landmark-Free, Deep, 3D Facial Expressions](https://mlsanity.com/paper/1802.00542/) (2018) - [Evaluation of Dense 3D Reconstruction from 2D Face Images in the Wild](https://mlsanity.com/paper/1803.05536/) (2018) - [MIMAMO Net: Integrating Micro- and Macro-motion for Video Emotion Recognition](https://mlsanity.com/paper/1911.09784/) (2019) - [Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition](https://mlsanity.com/paper/2103.13372/) (2021) - [A Sparse and Locally Coherent Morphable Face Model for Dense Semantic Correspondence Across Heterogeneous 3D Faces](https://mlsanity.com/paper/2006.03840/) (2020) - [Joint Action Unit localisation and intensity estimation through heatmap regression](https://mlsanity.com/paper/1805.03487/) (2018) - [A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation](https://mlsanity.com/paper/2004.06657/) (2020) - [DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Coding](https://mlsanity.com/paper/1704.02206/) (2017) - [Real-time Facial Expression Recognition "In The Wild'' by Disentangling 3D Expression from Identity](https://mlsanity.com/paper/2005.05509/) (2020) - [Impact of facial landmark localization on facial expression recognition](https://mlsanity.com/paper/1905.10784/) (2019) - [Linear Disentangled Representation Learning for Facial Actions](https://mlsanity.com/paper/1701.03102/) (2017) - [Variational Gaussian Process Auto-Encoder for Ordinal Prediction of Facial Action Units](https://mlsanity.com/paper/1608.04664/) (2016) ## Built on by (most cited first, 4 of 4) - [EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars](https://mlsanity.com/paper/2404.19110/) (2024) - [Towards Consistent and Controllable Image Synthesis for Face Editing](https://mlsanity.com/paper/2502.02465/) (2025) - [FIELDS: Face reconstruction with accurate Inference of Expression using Learning with Direct Supervision](https://mlsanity.com/paper/2511.21245/) (2025) - [PoreTrack3D: A Benchmark for Dynamic 3D Gaussian Splatting in Pore-Scale Facial Trajectory Tracking](https://mlsanity.com/paper/2512.02648/) (2025) ## Abstract (arXiv) Recognising continuous emotions and action unit (AU) intensities from face videos requires a spatial and temporal understanding of expression dynamics. Existing works primarily rely on 2D face appearances to extract such dynamics. This work focuses on a promising alternative based on parametric 3D face shape alignment models, which disentangle different factors of variation, including expression-induced shape variations. We aim to understand how expressive 3D face shapes are in estimating valence-arousal and AU intensities compared to the state-of-the-art 2D appearance-based models. We benchmark four recent 3D face alignment models: ExpNet, 3DDFA-V2, DECA, and EMOCA. In valence-arousal estimation, expression features of 3D face models consistently surpassed previous works and yielded an average concordance correlation of .739 and .574 on SEWA and AVEC 2019 CES corpora, respectively. We also study how 3D face shapes performed on AU intensity estimation on BP4D and DISFA datasets, and report that 3D face features were on par with 2D appearance features in AUs 4, 6, 10, 12, and 25, but not the entire set of AUs. To understand this discrepancy, we conduct a correspondence analysis between valence-arousal and AUs, which points out that accurate prediction of valence-arousal may require the knowledge of only a few AUs.