# A lightweight 3D dense facial landmark estimation model from position map data

arXiv: https://arxiv.org/abs/2308.15170 · HTML: https://mlsanity.com/paper/2308.15170/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 29 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://mlsanity.com/paper/1907.06724/) (2019)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies](https://mlsanity.com/paper/2010.05562/) (2020)
- [DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild](https://mlsanity.com/paper/1803.02188/) (2018)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses](https://mlsanity.com/paper/1707.05653/) (2017)
- [Interspecies Knowledge Transfer for Facial Keypoint Detection](https://mlsanity.com/paper/1704.04023/) (2017)
- [3D Morphable Models as Spatial Transformer Networks](https://mlsanity.com/paper/1708.07199/) (2017)

## Built on by (most cited first, 1 of 1)

- [Infinite 3D Landmarks: Improving Continuous 2D Facial Landmark Detection](https://mlsanity.com/paper/2405.20117/) (2024)

## Abstract (arXiv)

The incorporation of 3D data in facial analysis tasks has gained popularity in recent years. Though it provides a more accurate and detailed representation of the human face, accruing 3D face data is more complex and expensive than 2D face images. Either one has to rely on expensive 3D scanners or depth sensors which are prone to noise. An alternative option is the reconstruction of 3D faces from uncalibrated 2D images in an unsupervised way without any ground truth 3D data. However, such approaches are computationally expensive and the learned model size is not suitable for mobile or other edge device applications. Predicting dense 3D landmarks over the whole face can overcome this issue. As there is no public dataset available containing dense landmarks, we propose a pipeline to create a dense keypoint training dataset containing 520 key points across the whole face from an existing facial position map data. We train a lightweight MobileNet-based regressor model with the generated data. As we do not have access to any evaluation dataset with dense landmarks in it we evaluate our model against the 68 keypoint detection task. Experimental results show that our trained model outperforms many of the existing methods in spite of its lower model size and minimal computational cost. Also, the qualitative evaluation shows the efficiency of our trained models in extreme head pose angles as well as other facial variations and occlusions.