# Weakly Supervised Learning for Facial Affective Behavior Analysis : A Review

arXiv: https://arxiv.org/abs/2101.09858 · HTML: https://mlsanity.com/paper/2101.09858/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 151 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://mlsanity.com/paper/1612.03365/) (2016)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [A Survey on Machine Learning from Few Samples](https://mlsanity.com/paper/2009.02653/) (2020)
- [Deep Weakly-Supervised Domain Adaptation for Pain Localization in Videos](https://mlsanity.com/paper/1910.08173/) (2019)
- [Deep Domain Adaptation for Ordinal Regression of Pain Intensity Estimation Using Weakly-Labelled Videos](https://mlsanity.com/paper/2008.06392/) (2020)
- [Multi-Instance Dynamic Ordinal Random Fields for Weakly-supervised Facial Behavior Analysis](https://mlsanity.com/paper/1803.00907/) (2018)
- [LOMo: Latent Ordinal Model for Facial Analysis in Videos](https://mlsanity.com/paper/1604.01500/) (2016)
- [Multi-instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation](https://mlsanity.com/paper/1609.01465/) (2016)
- [Semi-supervised Facial Action Unit Intensity Estimation with Contrastive Learning](https://mlsanity.com/paper/2011.01864/) (2020)

## Built on by (most cited first, 2 of 2)

- [Knowing When Not to Answer: Abstention and Refusal Reasoning in Vision--Language Models](https://mlsanity.com/paper/2609.05540/) (2026)
- [BEACON: Behavior and Appearance Control for Subject-Specific Video Generation](https://mlsanity.com/paper/2609.13264/) (2026)

## Abstract (arXiv)

Recent advances in deep learning (DL) and computational capacity have enabled facial affective behavior analysis (FABA) to progress from static images captured in controlled settings to fine-grained analysis of facial expressions in real-world video data. However, training accurate DL models for FABA typically requires large-scale, expert-annotated datasets, which are costly to obtain and inherently noisy due to the ambiguity of labeling subtle facial expressions and action units (AUs). To mitigate these challenges, weakly supervised learning (WSL) has emerged as a promising paradigm for training models with weak annotations. In this paper, we present a structured taxonomy of WSL scenarios for FABA, organized according to the type of weak annotation and the specific affective task. Building on this taxonomy, we provide a critical synthesis of representative WSL methods for both classification (expression and AU recognition) and regression (expression and AU intensity estimation) tasks, focusing on their core methodological ideas, strengths, and limitations. Furthermore, we systematically summarize the comparative performance of WSL approaches along with widely adopted experimental setups and evaluation protocols. Our critical assessment identifies key challenges and future research directions, including the need for efficient adaptation of foundation models and for the development of robust, scalable FABA systems suitable for real-world applications.