# Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/1910.11012 · HTML: https://mlsanity.com/paper/1910.11012/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2019.

## Built on by (most cited first, 26 of 26)

- [Multi-Label Learning from Single Positive Labels](https://mlsanity.com/paper/2106.09708/) (2021)
- [Facial Affective Behavior Analysis with Instruction Tuning](https://mlsanity.com/paper/2404.05052/) (2024)
- [Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation](https://mlsanity.com/paper/2210.15878/) (2022)
- [Graph-based Facial Affect Analysis: A Review](https://mlsanity.com/paper/2103.15599/) (2021)
- [CLIP-MUSED: CLIP-Guided Multi-Subject Visual Neural Information Semantic Decoding](https://mlsanity.com/paper/2402.08994/) (2024)
- [Affective Behaviour Analysis Using Pretrained Model with Facial Priori](https://mlsanity.com/paper/2207.11679/) (2022)
- [The Lighter The Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning](https://mlsanity.com/paper/2206.14413/) (2022)
- [G2NetPL: Generic Game-Theoretic Network for Partial-Label Image Classification](https://mlsanity.com/paper/2210.11469/) (2022)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [PLMCL: Partial-Label Momentum Curriculum Learning for Multi-Label Image Classification](https://mlsanity.com/paper/2208.09999/) (2022)
- [Deep Partial Multi-Label Learning with Graph Disambiguation](https://mlsanity.com/paper/2305.05882/) (2023)
- [Task-adaptive Q-Face](https://mlsanity.com/paper/2405.09059/) (2024)
- [Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/2102.12154/) (2021)
- [Weakly Supervised Regional and Temporal Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/2204.00379/) (2022)
- [A unified framework based on graph consensus term for multi-view learning](https://mlsanity.com/paper/2105.11781/) (2021)
- [SCGG: A Deep Structure-Conditioned Graph Generative Model](https://mlsanity.com/paper/2209.09681/) (2022)
- [Learning Contrastive Feature Representations for Facial Action Unit Detection](https://mlsanity.com/paper/2402.06165/) (2024)
- [Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition](https://mlsanity.com/paper/2107.14399/) (2021)
- [Automatic Facial Paralysis Estimation with Facial Action Units](https://mlsanity.com/paper/2203.01800/) (2022)
- [Knowledge-Spreader: Learning Facial Action Unit Dynamics with Extremely Limited Labels](https://mlsanity.com/paper/2203.16678/) (2022)
- [GaFET: Learning Geometry-aware Facial Expression Translation from In-The-Wild Images](https://mlsanity.com/paper/2308.03413/) (2023)
- [Improving Long-tailed Object Detection with Image-Level Supervision by Multi-Task Collaborative Learning](https://mlsanity.com/paper/2210.05568/) (2022)
- [Global-to-local Expression-aware Embeddings for Facial Action Unit Detection](https://mlsanity.com/paper/2210.15160/) (2022)
- [Multi-view Action Recognition via Directed Gromov-Wasserstein Discrepancy](https://mlsanity.com/paper/2405.01337/) (2024)
- [Emotion Knowledge Enhancement for Vision Large Language Models: A Self-Verification Approach for High-Quality Emotion Instruction Data Generation](https://mlsanity.com/paper/2505.18168/) (2025)
- [AUCH-Net: Action Unit-Based Consistency-Aware Hypergraph Network for Cross-Domain Few-Shot Facial Expression Recognition](https://mlsanity.com/paper/2607.21004/) (2026)

## Abstract (arXiv)

Facial action units (AUs) recognition is essential for emotion analysis and has been widely applied in mental state analysis. Existing work on AU recognition usually requires big face dataset with AU labels; however, manual AU annotation requires expertise and can be time-consuming. In this work, we propose a semi-supervised approach for AU recognition utilizing a large number of web face images without AU labels and a relatively small face dataset with AU annotations inspired by the co-training methods. Unlike traditional co-training methods that require provided multi-view features and model re-training, we propose a novel co-training method, namely multi-label co-regularization, for semi-supervised facial AU recognition. Two deep neural networks are utilized to generate multi-view features for both labeled and unlabeled face images, and a multi-view loss is designed to enforce the two feature generators to get conditional independent representations. In order to constrain the prediction consistency of the two views, we further propose a multi-label co-regularization loss by minimizing the distance of the predicted AU probability distributions of two views. In addition, prior knowledge of the relationship between individual AUs is embedded through a graph convolutional network (GCN) for exploiting useful information from the big unlabeled dataset. Experiments on several benchmarks show that the proposed approach can effectively leverage large datasets of face images without AU labels to improve the AU recognition accuracy and outperform the state-of-the-art semi-supervised AU recognition methods.