# Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees

arXiv: https://arxiv.org/abs/1902.01831 · HTML: https://mlsanity.com/paper/1902.01831/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2019.

## Built on by (most cited first, 8 of 8)

- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [Multi-task head pose estimation in-the-wild](https://mlsanity.com/paper/2202.02299/) (2022)
- [Shape Preserving Facial Landmarks with Graph Attention Networks](https://mlsanity.com/paper/2210.07233/) (2022)
- [Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi-Person Human Pose Estimation](https://mlsanity.com/paper/2111.08557/) (2021)
- [Facial Landmark Points Detection Using Knowledge Distillation-Based Neural Networks](https://mlsanity.com/paper/2111.07047/) (2021)
- [Reasoning Structural Relation for Occlusion-Robust Facial Landmark Localization](https://mlsanity.com/paper/2112.10087/) (2021)
- [FineNet: Frame Interpolation and Enhancement for Face Video Deblurring](https://mlsanity.com/paper/2103.00871/) (2021)
- [AUGlasses: Continuous Action Unit based Facial Reconstruction with Low-power IMUs on Smart Glasses](https://mlsanity.com/paper/2405.13289/) (2024)

## Abstract (arXiv)

Face alignment algorithms locate a set of landmark points in images of faces taken in unrestricted situations. State-of-the-art approaches typically fail or lose accuracy in the presence of occlusions, strong deformations, large pose variations and ambiguous configurations. In this paper we present 3DDE, a robust and efficient face alignment algorithm based on a coarse-to-fine cascade of ensembles of regression trees. It is initialized by robustly fitting a 3D face model to the probability maps produced by a convolutional neural network. With this initialization we address self-occlusions and large face rotations. Further, the regressor implicitly imposes a prior face shape on the solution, addressing occlusions and ambiguous face configurations. Its coarse-to-fine structure tackles the combinatorial explosion of parts deformation. In the experiments performed, 3DDE improves the state-of-the-art in 300W, COFW, AFLW and WFLW data sets. Finally, we perform cross-dataset experiments that reveal the existence of a significant data set bias in these benchmarks.