# Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization

arXiv: https://arxiv.org/abs/2112.10087 · HTML: https://mlsanity.com/paper/2112.10087/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 19 Dec 2021.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors](https://mlsanity.com/paper/1807.00966/) (2018)
- [3FabRec: Fast Few-shot Face alignment by Reconstruction](https://mlsanity.com/paper/1911.10448/) (2019)
- [Learning Deep Representation for Face Alignment with Auxiliary Attributes](https://mlsanity.com/paper/1408.3967/) (2014)
- [Example-Guided Style Consistent Image Synthesis from Semantic Labeling](https://mlsanity.com/paper/1906.01314/) (2019)
- [3D Face Reconstruction with Geometry Details from a Single Image](https://mlsanity.com/paper/1702.05619/) (2017)
- [Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition](https://mlsanity.com/paper/1708.02734/) (2017)
- [Geometry-Aware Face Completion and Editing](https://mlsanity.com/paper/1809.02967/) (2018)
- [Towards Highly Accurate and Stable Face Alignment for High-Resolution Videos](https://mlsanity.com/paper/1811.00342/) (2018)
- [Disentangling 3D Pose in A Dendritic CNN for Unconstrained 2D Face Alignment](https://mlsanity.com/paper/1802.06713/) (2018)
- [Semantic Alignment: Finding Semantically Consistent Ground-truth for Facial Landmark Detection](https://mlsanity.com/paper/1903.10661/) (2019)
- [Towards Omni-Supervised Face Alignment for Large Scale Unlabeled Videos](https://mlsanity.com/paper/1912.07243/) (2019)
- [FAB: A Robust Facial Landmark Detection Framework for Motion-Blurred Videos](https://mlsanity.com/paper/1910.12100/) (2019)
- [Pairwise Relational Networks for Face Recognition](https://mlsanity.com/paper/1808.04976/) (2018)
- [PropagationNet: Propagate Points to Curve to Learn Structure Information](https://mlsanity.com/paper/2006.14308/) (2020)
- [A Recurrent Encoder-Decoder Network for Sequential Face Alignment](https://mlsanity.com/paper/1608.05477/) (2016)
- [Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees](https://mlsanity.com/paper/1902.01831/) (2019)
- [Occlusion Coherence: Detecting and Localizing Occluded Faces](https://mlsanity.com/paper/1506.08347/) (2015)
- [Cascaded Continuous Regression for Real-time Incremental Face Tracking](https://mlsanity.com/paper/1608.01137/) (2016)

## Built on by (most cited first, 2 of 2)

- [Precise Facial Landmark Detection by Dynamic Semantic Aggregation Transformer](https://mlsanity.com/paper/2412.00740/) (2024)
- [1DFormer: a Transformer Architecture Learning 1D Landmark Representations for Facial Landmark Tracking](https://mlsanity.com/paper/2311.00241/) (2023)

## Abstract (arXiv)

In facial landmark localization tasks, various occlusions heavily degrade the localization accuracy due to the partial observability of facial features. This paper proposes a structural relation network (SRN) for occlusion-robust landmark localization. Unlike most existing methods that simply exploit the shape constraint, the proposed SRN aims to capture the structural relations among different facial components. These relations can be considered a more powerful shape constraint against occlusion. To achieve this, a hierarchical structural relation module (HSRM) is designed to hierarchically reason the structural relations that represent both long- and short-distance spatial dependencies. Compared with existing network architectures, HSRM can efficiently model the spatial relations by leveraging its geometry-aware network architecture, which reduces the semantic ambiguity caused by occlusion. Moreover, the SRN augments the training data by synthesizing occluded faces. To further extend our SRN for occluded video data, we formulate the occluded face synthesis as a Markov decision process (MDP). Specifically, it plans the movement of the dynamic occlusion based on an accumulated reward associated with the performance degradation of the pre-trained SRN. This procedure augments hard samples for robust facial landmark tracking. Extensive experimental results indicate that the proposed method achieves outstanding performance on occluded and masked faces. Code is available at https://github.com/zhuccly/SRN.