# LDDMM-Face: Large Deformation Diffeomorphic Metric Learning for Flexible and Consistent Face Alignment

arXiv: https://arxiv.org/abs/2108.00690 · HTML: https://mlsanity.com/paper/2108.00690/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 2 Aug 2021.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [Learning Detailed Face Reconstruction from a Single Image](https://mlsanity.com/paper/1611.05053/) (2016)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [DeCaFA: Deep Convolutional Cascade for Face Alignment In The Wild](https://mlsanity.com/paper/1904.02549/) (2019)
- [Deep Alignment Network: A convolutional neural network for robust face alignment](https://mlsanity.com/paper/1706.01789/) (2017)
- [Aggregation via Separation: Boosting Facial Landmark Detector with Semi-Supervised Style Translation](https://mlsanity.com/paper/1908.06440/) (2019)
- [3FabRec: Fast Few-shot Face alignment by Reconstruction](https://mlsanity.com/paper/1911.10448/) (2019)
- [Learning Deep Representation for Face Alignment with Auxiliary Attributes](https://mlsanity.com/paper/1408.3967/) (2014)
- [Teacher Supervises Students How to Learn From Partially Labeled Images for Facial Landmark Detection](https://mlsanity.com/paper/1908.02116/) (2019)
- [Structured Feature Learning for Pose Estimation](https://mlsanity.com/paper/1603.09065/) (2016)
- [Two-stage Convolutional Part Heatmap Regression for the 1st 3D Face Alignment in the Wild (3DFAW) Challenge](https://mlsanity.com/paper/1609.09545/) (2016)
- [Occlusion Coherence: Detecting and Localizing Occluded Faces](https://mlsanity.com/paper/1506.08347/) (2015)
- [Transferring Landmark Annotations for Cross-Dataset Face Alignment](https://mlsanity.com/paper/1409.0602/) (2014)
- [Robust Face Alignment Using a Mixture of Invariant Experts](https://mlsanity.com/paper/1511.04404/) (2015)

## Abstract (arXiv)

We innovatively propose a flexible and consistent face alignment framework, LDDMM-Face, the key contribution of which is a deformation layer that naturally embeds facial geometry in a diffeomorphic way. Instead of predicting facial landmarks via heatmap or coordinate regression, we formulate this task in a diffeomorphic registration manner and predict momenta that uniquely parameterize the deformation between initial boundary and true boundary, and then perform large deformation diffeomorphic metric mapping (LDDMM) simultaneously for curve and landmark to localize the facial landmarks. Due to the embedding of LDDMM into a deep network, LDDMM-Face can consistently annotate facial landmarks without ambiguity and flexibly handle various annotation schemes, and can even predict dense annotations from sparse ones. Our method can be easily integrated into various face alignment networks. We extensively evaluate LDDMM-Face on four benchmark datasets: 300W, WFLW, HELEN and COFW-68. LDDMM-Face is comparable or superior to state-of-the-art methods for traditional within-dataset and same-annotation settings, but truly distinguishes itself with outstanding performance when dealing with weakly-supervised learning (partial-to-full), challenging cases (e.g., occluded faces), and different training and prediction datasets. In addition, LDDMM-Face shows promising results on the most challenging task of predicting across datasets with different annotation schemes.