Knowing When to Quit: Selective Cascaded Regression with Patch Attention for Real-Time Face Alignment
Not on the map. Why?
Abstract (arXiv)
Facial landmarks (FLM) estimation is a critical component in many face-related applications. In this work, we aim to optimize for both accuracy and speed and explore the trade-off between them. Our key observation is that not all faces are created equal. Frontal faces with neutral expressions converge faster than faces with extreme poses or expressions. To differentiate among samples, we train our model to predict the regression error after each iteration. If the current iteration is accurate enough, we stop iterating, saving redundant iterations while keeping the accuracy in check. We also observe that as neighboring patches overlap, we can infer all facial landmarks (FLMs) with only a small number of patches without a major accuracy sacrifice. Architecturally, we offer a multi-scale, patch-based, lightweight feature extractor with a fine-grained local patch attention module, which computes a patch weighting according to the information in the patch itself and enhances the expressive power of the patch features. We analyze the patch attention data to infer where the model is attending when regressing facial landmarks and compare it to face attention in humans. Our model runs in real-time on a mobile device GPU, with 95 Mega Multiply-Add (MMA) operations, outperforming all state-of-the-art methods under 1000 MMA, with a normalized mean error of 8.16 on the 300W challenging dataset.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 21 of its 50 references that are in the map. First posted on arXiv 1 Aug 2021.
Builds on
21 of 50 references.
References · 21
- 2017Attention Is All You Needcited by 60,106
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,318
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,786
- 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,418
- 2019Stand-Alone Self-Attention in Vision Modelscited by 542
- 2015Attention-Based Models for Speech Recognitioncited by 400
- 2014Recurrent Models of Visual Attentioncited by 344
- 2015WIDER FACE: A Face Detection Benchmarkcited by 298
- 2016Image Captioning with Semantic Attentioncited by 260
- 2019High-Resolution Representations for Labeling Pixels and Regionscited by 211
- 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
- 2018Look at Boundary: A Boundary-Aware Face Alignment Algorithmcited by 137
9 more
- 2019Adaptive Wing Loss for Robust Face Alignment via Heatmap Regressioncited by 128
- 2017Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networkscited by 108
- 2020LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihoodcited by 87
- 2019DeCaFA: Deep Convolutional Cascade for Face Alignment In The Wildcited by 46
- 2017Deep Alignment Network: A convolutional neural network for robust face alignmentcited by 44
- 2017Dense Face Alignmentcited by 24
- 2019Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detectioncited by 16
- 2018Facial Landmarks Localization using Cascaded Neural Networkscited by 4
- 2018Facial Landmarks Detection by Self-Iterative Regression based Landmarks-Attention Networkcited by 3
Built on by
Citing papers per year: 2021: 1.
1 paper.