# Deep Structure Inference Network for Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/1803.05873 · HTML: https://mlsanity.com/paper/1803.05873/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2018.

## Built on by (most cited first, 30 of 38)

- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [Facial Affective Behavior Analysis with Instruction Tuning](https://mlsanity.com/paper/2404.05052/) (2024)
- [GPT as Psychologist? Preliminary Evaluations for GPT-4V on Visual Affective Computing](https://mlsanity.com/paper/2403.05916/) (2024)
- [Facial Action Unit Detection and Intensity Estimation from Self-supervised Representation](https://mlsanity.com/paper/2210.15878/) (2022)
- [Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding](https://mlsanity.com/paper/2304.00058/) (2023)
- [Graph-based Facial Affect Analysis: A Review](https://mlsanity.com/paper/2103.15599/) (2021)
- [Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2308.07770/) (2023)
- [BigSmall: Efficient Multi-Task Learning for Disparate Spatial and Temporal Physiological Measurements](https://mlsanity.com/paper/2303.11573/) (2023)
- [Causal Intervention for Subject-Deconfounded Facial Action Unit Recognition](https://mlsanity.com/paper/2204.07935/) (2022)
- [Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection](https://mlsanity.com/paper/2209.12244/) (2022)
- [Knowledge-augmented Deep Learning and Its Applications: A Survey](https://mlsanity.com/paper/2212.00017/) (2022)
- [Multi-label Transformer for Action Unit Detection](https://mlsanity.com/paper/2203.12531/) (2022)
- [Meta Auxiliary Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2105.06620/) (2021)
- [Multi-Order Networks for Action Unit Detection](https://mlsanity.com/paper/2202.00446/) (2022)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [Facial Action Unit Detection by Adaptively Constraining Self-Attention and Causally Deconfounding Sample](https://mlsanity.com/paper/2410.01251/) (2024)
- [AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkit](https://mlsanity.com/paper/2202.12059/) (2022)
- [Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformers](https://mlsanity.com/paper/2203.11441/) (2022)
- [FAN-Trans: Online Knowledge Distillation for Facial Action Unit Detection](https://mlsanity.com/paper/2211.06143/) (2022)
- [Contrastive Learning of Person-independent Representations for Facial Action Unit Detection](https://mlsanity.com/paper/2403.03400/) (2024)
- [OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis](https://mlsanity.com/paper/2506.02891/) (2025)
- [Your "Attention" Deserves Attention: A Self-Diversified Multi-Channel Attention for Facial Action Analysis](https://mlsanity.com/paper/2203.12570/) (2022)
- [Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/2102.12154/) (2021)
- [Weakly Supervised Regional and Temporal Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/2204.00379/) (2022)
- [Hierarchical Vision-Language Interaction for Facial Action Unit Detection](https://mlsanity.com/paper/2602.14425/) (2026)
- [Occlusion-Robust FAU Recognition by Mining Latent Space of Masked Autoencoders](https://mlsanity.com/paper/2212.04029/) (2022)
- [Learning Contrastive Feature Representations for Facial Action Unit Detection](https://mlsanity.com/paper/2402.06165/) (2024)
- [Towards Unified Facial Action Unit Recognition Framework by Large Language Models](https://mlsanity.com/paper/2409.08444/) (2024)
- [Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition](https://mlsanity.com/paper/2107.14399/) (2021)
- [Automatic Facial Paralysis Estimation with Facial Action Units](https://mlsanity.com/paper/2203.01800/) (2022)

## Abstract (arXiv)

Facial expressions are combinations of basic components called Action Units (AU). Recognizing AUs is key for developing general facial expression analysis. In recent years, most efforts in automatic AU recognition have been dedicated to learning combinations of local features and to exploiting correlations between Action Units. In this paper, we propose a deep neural architecture that tackles both problems by combining learned local and global features in its initial stages and replicating a message passing algorithm between classes similar to a graphical model inference approach in later stages. We show that by training the model end-to-end with increased supervision we improve state-of-the-art by 5.3% and 8.2% performance on BP4D and DISFA datasets, respectively.