# Facial Affect Analysis: Learning from Synthetic Data & Multi-Task Learning Challenges

arXiv: https://arxiv.org/abs/2207.09748 · HTML: https://mlsanity.com/paper/2207.09748/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 21 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 14 of 14)

- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [ExprGAN: Facial Expression Editing with Controllable Expression Intensity](https://mlsanity.com/paper/1709.03842/) (2017)
- [Photorealistic Facial Synthesis in the Dimensional Affect Space](https://mlsanity.com/paper/1811.08004/) (2018)
- [Multiple Emotion Descriptors Estimation at the ABAW3 Challenge](https://mlsanity.com/paper/2203.12845/) (2022)

## Built on by (most cited first, 10 of 10)

- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [HSEmotion Team at the 7th ABAW Challenge: Multi-Task Learning and Compound Facial Expression Recognition](https://mlsanity.com/paper/2407.13184/) (2024)
- [Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection](https://mlsanity.com/paper/2303.08545/) (2023)
- [Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers](https://mlsanity.com/paper/2303.09164/) (2023)
- [TempT: Temporal consistency for Test-time adaptation](https://mlsanity.com/paper/2303.10536/) (2023)
- [ABAW : Facial Expression Recognition in the wild](https://mlsanity.com/paper/2303.09785/) (2023)
- [A Dual Branch Network for Emotional Reaction Intensity Estimation](https://mlsanity.com/paper/2303.09210/) (2023)
- [HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognition](https://mlsanity.com/paper/2607.12774/) (2026)

## Abstract (arXiv)

Facial affect analysis remains a challenging task with its setting transitioned from lab-controlled to in-the-wild situations. In this paper, we present novel frameworks to handle the two challenges in the 4th Affective Behavior Analysis In-The-Wild (ABAW) competition: i) Multi-Task-Learning (MTL) Challenge and ii) Learning from Synthetic Data (LSD) Challenge. For MTL challenge, we adopt the SMM-EmotionNet with a better ensemble strategy of feature vectors. For LSD challenge, we propose respective methods to combat the problems of single labels, imbalanced distribution, fine-tuning limitations, and choice of model architectures. Experimental results on the official validation sets from the competition demonstrated that our proposed approaches outperformed baselines by a large margin. The code is available at https://github.com/sylyoung/ABAW4-HUST-ANT.