# 3D face reconstruction with dense landmarks

arXiv: https://arxiv.org/abs/2204.02776 · HTML: https://mlsanity.com/paper/2204.02776/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 78 references that are in the map.
- First posted on arXiv 6 Apr 2022.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Learning an Animatable Detailed 3D Face Model from In-The-Wild Images](https://mlsanity.com/paper/2012.04012/) (2020)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018)
- [Morphable Face Models - An Open Framework](https://mlsanity.com/paper/1709.08398/) (2017)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz](https://mlsanity.com/paper/1712.02859/) (2017)
- [Towards High-fidelity Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1904.04933/) (2019)
- [LUVLi Face Alignment: Estimating Landmarks' Location, Uncertainty, and Visibility Likelihood](https://mlsanity.com/paper/2004.02980/) (2020)
- [FML: Face Model Learning from Videos](https://mlsanity.com/paper/1812.07603/) (2018)
- [Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency](https://mlsanity.com/paper/2007.12494/) (2020)
- [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://mlsanity.com/paper/1907.06724/) (2019)
- [Learning Detailed Face Reconstruction from a Single Image](https://mlsanity.com/paper/1611.05053/) (2016)
- [3D Face Reconstruction by Learning from Synthetic Data](https://mlsanity.com/paper/1609.04387/) (2016)
- [Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation](https://mlsanity.com/paper/1703.10131/) (2017)

## Built on by (most cited first, 30 of 70)

- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [DigiFace-1M: 1 Million Digital Face Images for Face Recognition](https://mlsanity.com/paper/2210.02579/) (2022)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation](https://mlsanity.com/paper/2212.08062/) (2022)
- [Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2209.10510/) (2022)
- [Real-Time Radiance Fields for Single-Image Portrait View Synthesis](https://mlsanity.com/paper/2305.02310/) (2023)
- [Portrait4D: Learning One-Shot 4D Head Avatar Synthesis using Synthetic Data](https://mlsanity.com/paper/2311.18729/) (2023)
- [STAR Loss: Reducing Semantic Ambiguity in Facial Landmark Detection](https://mlsanity.com/paper/2306.02763/) (2023)
- [VolTeMorph: Realtime, Controllable and Generalisable Animation of Volumetric Representations](https://mlsanity.com/paper/2208.00949/) (2022)
- [Instant Multi-View Head Capture through Learnable Registration](https://mlsanity.com/paper/2306.07437/) (2023)
- [FaceXFormer: A Unified Transformer for Facial Analysis](https://mlsanity.com/paper/2403.12960/) (2024)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)
- [DaGAN++: Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2305.06225/) (2023)
- [3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow](https://mlsanity.com/paper/2404.09819/) (2024)
- [Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction](https://mlsanity.com/paper/2505.00615/) (2025)
- [SimpleEgo: Predicting Probabilistic Body Pose from Egocentric Cameras](https://mlsanity.com/paper/2401.14785/) (2024)
- [Text-Guided 3D Face Synthesis -- From Generation to Editing](https://mlsanity.com/paper/2312.00375/) (2023)
- [DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image](https://mlsanity.com/paper/2406.17988/) (2024)
- [Body Knowledge and Uncertainty Modeling for Monocular 3D Human Body Reconstruction](https://mlsanity.com/paper/2308.00799/) (2023)
- [Make-A-Character: High Quality Text-to-3D Character Generation within Minutes](https://mlsanity.com/paper/2312.15430/) (2023)
- [Topo4D: Topology-Preserving Gaussian Splatting for High-Fidelity 4D Head Capture](https://mlsanity.com/paper/2406.00440/) (2024)
- [SHeaP: Self-Supervised Head Geometry Predictor Learned via 2D Gaussians](https://mlsanity.com/paper/2504.12292/) (2025)
- [T-FAKE: Synthesizing Thermal Images for Facial Landmarking](https://mlsanity.com/paper/2408.15127/) (2024)
- [Few-shot Geometry-Aware Keypoint Localization](https://mlsanity.com/paper/2303.17216/) (2023)
- [ASM: Adaptive Skinning Model for High-Quality 3D Face Modeling](https://mlsanity.com/paper/2304.09423/) (2023)
- [BlendFields: Few-Shot Example-Driven Facial Modeling](https://mlsanity.com/paper/2305.07514/) (2023)
- [DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment](https://mlsanity.com/paper/2305.11522/) (2023)
- [A Perceptual Shape Loss for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2310.19580/) (2023)
- [SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation](https://mlsanity.com/paper/2405.18322/) (2024)
- [VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2409.09740/) (2024)

## Abstract (arXiv)

Landmarks often play a key role in face analysis, but many aspects of identity or expression cannot be represented by sparse landmarks alone. Thus, in order to reconstruct faces more accurately, landmarks are often combined with additional signals like depth images or techniques like differentiable rendering. Can we keep things simple by just using more landmarks? In answer, we present the first method that accurately predicts 10x as many landmarks as usual, covering the whole head, including the eyes and teeth. This is accomplished using synthetic training data, which guarantees perfect landmark annotations. By fitting a morphable model to these dense landmarks, we achieve state-of-the-art results for monocular 3D face reconstruction in the wild. We show that dense landmarks are an ideal signal for integrating face shape information across frames by demonstrating accurate and expressive facial performance capture in both monocular and multi-view scenarios. This approach is also highly efficient: we can predict dense landmarks and fit our 3D face model at over 150FPS on a single CPU thread. Please see our website: https://microsoft.github.io/DenseLandmarks/.