Automatic Facial Paralysis Estimation with Facial Action Units
Not on the map. Why?
Abstract (arXiv)
Facial palsy is unilateral facial nerve weakness or paralysis of rapid onset with unknown causes. Automatically estimating facial palsy severeness can be helpful for the diagnosis and treatment of people suffering from it across the world. In this work, we develop and experiment with a novel model for estimating facial palsy severity. For this, an effective Facial Action Units (AU) detection technique is incorporated into our model, where AUs refer to a unique set of facial muscle movements used to describe almost every anatomically possible facial expression. In this paper, we propose a novel Adaptive Local-Global Relational Network (ALGRNet) for facial AU detection and use it to classify facial paralysis severity. ALGRNet mainly consists of three main novel structures: (i) an adaptive region learning module that learns the adaptive muscle regions based on the detected landmarks; (ii) a skip-BiLSTM that models the latent relationships among local AUs; and (iii) a feature fusion&refining module that investigates the complementary between the local and global face. Quantitative results on two AU benchmarks, i.e., BP4D and DISFA, demonstrate our ALGRNet can achieve promising AU detection accuracy. We further demonstrate the effectiveness of its application to facial paralysis estimation by migrating ALGRNet to a facial paralysis dataset collected and annotated by medical professionals.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 19 of its 71 references that are in the map. First posted on arXiv 3 Mar 2022.
Builds on
19 of 71 references.
References · 19
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
- 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
- 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 319
- 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 305
- 2020JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attentioncited by 62
- 2018Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignmentcited by 55
- 2019Semantic Relationships Guided Representation Learning for Facial Action Unit Recognitioncited by 53
- 2018Facial Action Unit Detection Using Attention and Relation Learningcited by 46
7 more
- 2018Deep Structure Inference Network for Facial Action Unit Recognitioncited by 38
- 2017Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusingcited by 28
- 2019Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognitioncited by 26
- 2021Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrievalcited by 23
- 2018AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detectioncited by 14
- 2020A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimationcited by 9
- 2017Learning Pain from Action Unit Combinations: A Weakly Supervised Approach via Multiple Instance Learningcited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.