# Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/2102.12154 · HTML: https://mlsanity.com/paper/2102.12154/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 24 Feb 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Deep Structure Inference Network for Facial Action Unit Recognition](https://mlsanity.com/paper/1803.05873/) (2018)
- [Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusing](https://mlsanity.com/paper/1704.03067/) (2017)
- [EAC-Net: A Region-based Deep Enhancing and Cropping Approach for Facial Action Unit Detection](https://mlsanity.com/paper/1702.02925/) (2017)
- [Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition](https://mlsanity.com/paper/1910.11012/) (2019)
- [Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection](https://mlsanity.com/paper/1910.10334/) (2019)
- [Deep Structured Learning for Facial Action Unit Intensity Estimation](https://mlsanity.com/paper/1704.04481/) (2017)

## Built on by (most cited first, 3 of 3)

- [Weakly Supervised Regional and Temporal Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/2204.00379/) (2022)
- [Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition](https://mlsanity.com/paper/2107.14399/) (2021)
- [Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction](https://mlsanity.com/paper/2310.05207/) (2023)

## Abstract (arXiv)

In facial action unit (AU) recognition tasks, regional feature learning and AU relation modeling are two effective aspects which are worth exploring. However, the limited representation capacity of regional features makes it difficult for relation models to embed AU relationship knowledge. In this paper, we propose a novel multi-level adaptive ROI and graph learning (MARGL) framework to tackle this problem. Specifically, an adaptive ROI learning module is designed to automatically adjust the location and size of the predefined AU regions. Meanwhile, besides relationship between AUs, there exists strong relevance between regional features across multiple levels of the backbone network as level-wise features focus on different aspects of representation. In order to incorporate the intra-level AU relation and inter-level AU regional relevance simultaneously, a multi-level AU relation graph is constructed and graph convolution is performed to further enhance AU regional features of each level. Experiments on BP4D and DISFA demonstrate the proposed MARGL significantly outperforms the previous state-of-the-art methods.