# SS-MFAR : Semi-supervised Multi-task Facial Affect Recognition

arXiv: https://arxiv.org/abs/2207.09012 · HTML: https://mlsanity.com/paper/2207.09012/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Towards Semi-Supervised Deep Facial Expression Recognition with An Adaptive Confidence Margin](https://mlsanity.com/paper/2203.12341/) (2022)
- [A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressions](https://mlsanity.com/paper/1811.07771/) (2018)
- [Causal affect prediction model using a facial image sequence](https://mlsanity.com/paper/2107.03886/) (2021)
- [Two-level Attention with Two-stage Multi-task Learning for Facial Emotion Recognition](https://mlsanity.com/paper/1811.12139/) (2018)
- [A Deep Network for Arousal-Valence Emotion Prediction with Acoustic-Visual Cues](https://mlsanity.com/paper/1805.00638/) (2018)

## Built on by (most cited first, 5 of 5)

- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges](https://mlsanity.com/paper/2303.01498/) (2023)
- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [A Shared Latent for Partially-Labeled Multi-Task Facial Affect Recognition](https://mlsanity.com/paper/2607.16285/) (2026)
- [Strength-Parity Ensembling with Parameter-Isolated Experts for Multi-Task Affect Recognition](https://mlsanity.com/paper/2607.16290/) (2026)

## Abstract (arXiv)

Automatic affect recognition has applications in many areas such as education, gaming, software development, automotives, medical care, etc. but it is non trivial task to achieve appreciable performance on in-the-wild data sets. In-the-wild data sets though represent real-world scenarios better than synthetic data sets, the former ones suffer from the problem of incomplete labels. Inspired by semi-supervised learning, in this paper, we introduce our submission to the Multi-Task-Learning Challenge at the 4th Affective Behavior Analysis in-the-wild (ABAW) 2022 Competition. The three tasks that are considered in this challenge are valence-arousal(VA) estimation, classification of expressions into 6 basic (anger, disgust, fear, happiness, sadness, surprise), neutral, and the 'other' category and 12 action units(AU) numbered AU-{1,2,4,6,7,10,12,15,23,24,25,26}. Our method Semi-supervised Multi-task Facial Affect Recognition titled SS-MFAR uses a deep residual network with task specific classifiers for each of the tasks along with adaptive thresholds for each expression class and semi-supervised learning for the incomplete labels. Source code is available at https://github.com/1980x/ABAW2022DMACS.