# Photorealistic Facial Synthesis in the Dimensional Affect Space

arXiv: https://arxiv.org/abs/1811.08004 · HTML: https://mlsanity.com/paper/1811.08004/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2018.

## Built on by (most cited first, 30 of 33)

- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [A Deep Neural Architecture for Harmonizing 3-D Input Data Analysis and Decision Making in Medical Imaging](https://mlsanity.com/paper/2303.00175/) (2023)
- [MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition](https://mlsanity.com/paper/2205.04442/) (2022)
- [Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans](https://mlsanity.com/paper/2403.02192/) (2024)
- [COVID-19 Computer-aided Diagnosis through AI-assisted CT Imaging Analysis: Deploying a Medical AI System](https://mlsanity.com/paper/2403.06242/) (2024)
- [AI-MIA: COVID-19 Detection & Severity Analysis through Medical Imaging](https://mlsanity.com/paper/2206.04732/) (2022)
- [BTDNet: a Multi-Modal Approach for Brain Tumor Radiogenomic Classification](https://mlsanity.com/paper/2310.03485/) (2023)
- [DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarios](https://mlsanity.com/paper/2506.05372/) (2025)
- [Multi-Task Learning Framework for Emotion Recognition in-the-wild](https://mlsanity.com/paper/2207.09373/) (2022)
- [Facial Affect Analysis: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.09748/) (2022)
- [Affective Behaviour Analysis Using Pretrained Model with Facial Priori](https://mlsanity.com/paper/2207.11679/) (2022)
- [Affective Behavior Analysis using Action Unit Relation Graph and Multi-task Cross Attention](https://mlsanity.com/paper/2207.10293/) (2022)
- [HSE-NN Team at the 4th ABAW Competition: Multi-task Emotion Recognition and Learning from Synthetic Images](https://mlsanity.com/paper/2207.09508/) (2022)
- [Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing for Facial Expression Recognition](https://mlsanity.com/paper/2207.13235/) (2022)
- [Learning from Synthetic Data: Facial Expression Classification based on Ensemble of Multi-task Networks](https://mlsanity.com/paper/2207.10025/) (2022)
- [Hybrid CNN-Transformer Model For Facial Affect Recognition In the ABAW4 Challenge](https://mlsanity.com/paper/2207.10201/) (2022)
- [Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation](https://mlsanity.com/paper/2407.15798/) (2024)
- [Multi-Task Learning for Emotion Descriptors Estimation at the fourth ABAW Challenge](https://mlsanity.com/paper/2207.09716/) (2022)
- [BYEL : Bootstrap Your Emotion Latent](https://mlsanity.com/paper/2207.10003/) (2022)
- [An Ensemble Approach for Multiple Emotion Descriptors Estimation Using Multi-task Learning](https://mlsanity.com/paper/2207.10878/) (2022)
- [Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge](https://mlsanity.com/paper/2207.11389/) (2022)
- [ABAW : Facial Expression Recognition in the wild](https://mlsanity.com/paper/2303.09785/) (2023)
- [Grounding Emotion Recognition with Visual Prototypes: VEGA -- Revisiting CLIP in MERC](https://mlsanity.com/paper/2508.06564/) (2025)

## Abstract (arXiv)

This paper presents a novel approach for synthesizing facial affect, which is based on our annotating 600,000 frames of the 4DFAB database in terms of valence and arousal. The input of this approach is a pair of these emotional state descriptors and a neutral 2D image of a person to whom the corresponding affect will be synthesized. Given this target pair, a set of 3D facial meshes is selected, which is used to build a blendshape model and generate the new facial affect. To synthesize the affect on the 2D neutral image, 3DMM fitting is performed and the reconstructed face is deformed to generate the target facial expressions. Last, the new face is rendered into the original image. Both qualitative and quantitative experimental studies illustrate the generation of realistic images, when the neutral image is sampled from a variety of well known databases, such as the Aff-Wild, AFEW, Multi-PIE, AFEW-VA, BU-3DFE, Bosphorus.