# Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/1904.09939 · HTML: https://mlsanity.com/paper/1904.09939/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2019.

## Built on by (most cited first, 30 of 53)

- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors](https://mlsanity.com/paper/2403.04697/) (2024)
- [Facial Affective Behavior Analysis with Instruction Tuning](https://mlsanity.com/paper/2404.05052/) (2024)
- [GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing](https://mlsanity.com/paper/2403.05916/) (2024)
- [Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition](https://mlsanity.com/paper/2103.08637/) (2021)
- [Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation](https://mlsanity.com/paper/2210.15878/) (2022)
- [Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding](https://mlsanity.com/paper/2304.00058/) (2023)
- [ExpLLM: Towards Chain of Thought for Facial Expression Recognition](https://mlsanity.com/paper/2409.02828/) (2024)
- [Graph-based Facial Affect Analysis: A Review](https://mlsanity.com/paper/2103.15599/) (2021)
- [Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2308.07770/) (2023)
- [Representation Learning and Identity Adversarial Training for Facial Behavior Understanding](https://mlsanity.com/paper/2407.11243/) (2024)
- [Causal Intervention for Subject-Deconfounded Facial Action Unit Recognition](https://mlsanity.com/paper/2204.07935/) (2022)
- [GRATIS: Deep Learning Graph Representation with Task-specific Topology and Multi-dimensional Edge Features](https://mlsanity.com/paper/2211.12482/) (2022)
- [Self-supervised Facial Action Unit Detection with Region and Relation Learning](https://mlsanity.com/paper/2303.05708/) (2023)
- [FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features](https://mlsanity.com/paper/2308.12380/) (2023)
- [Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection](https://mlsanity.com/paper/2209.12244/) (2022)
- [Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition](https://mlsanity.com/paper/2404.06443/) (2024)
- [Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning](https://mlsanity.com/paper/2408.00644/) (2024)
- [Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection](https://mlsanity.com/paper/2303.10644/) (2023)
- [AU-LLM: Micro-Expression Action Unit Detection via Enhanced LLM-Based Feature Fusion](https://mlsanity.com/paper/2507.21778/) (2025)
- [Meta Auxiliary Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2105.06620/) (2021)
- [Multi-Order Networks for Action Unit Detection](https://mlsanity.com/paper/2202.00446/) (2022)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample](https://mlsanity.com/paper/2410.01251/) (2024)
- [DeepFN: Towards Generalizable Facial Action Unit Recognition with Deep Face Normalization](https://mlsanity.com/paper/2103.02484/) (2021)
- [Unconstrained Face Sketch Synthesis via Perception-Adaptive Network and A New Benchmark](https://mlsanity.com/paper/2112.01019/) (2021)
- [AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkit](https://mlsanity.com/paper/2202.12059/) (2022)
- [Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformers](https://mlsanity.com/paper/2203.11441/) (2022)
- [FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection](https://mlsanity.com/paper/2211.06143/) (2022)
- [Contrastive Learning of Person-independent Representations for Facial Action Unit Detection](https://mlsanity.com/paper/2403.03400/) (2024)

## Abstract (arXiv)

Facial action unit (AU) recognition is a crucial task for facial expressions analysis and has attracted extensive attention in the field of artificial intelligence and computer vision. Existing works have either focused on designing or learning complex regional feature representations, or delved into various types of AU relationship modeling. Albeit with varying degrees of progress, it is still arduous for existing methods to handle complex situations. In this paper, we investigate how to integrate the semantic relationship propagation between AUs in a deep neural network framework to enhance the feature representation of facial regions, and propose an AU semantic relationship embedded representation learning (SRERL) framework. Specifically, by analyzing the symbiosis and mutual exclusion of AUs in various facial expressions, we organize the facial AUs in the form of structured knowledge-graph and integrate a Gated Graph Neural Network (GGNN) in a multi-scale CNN framework to propagate node information through the graph for generating enhanced AU representation. As the learned feature involves both the appearance characteristics and the AU relationship reasoning, the proposed model is more robust and can cope with more challenging cases, e.g., illumination change and partial occlusion. Extensive experiments on the two public benchmarks demonstrate that our method outperforms the previous work and achieves state of the art performance.