# Towards Metrical Reconstruction of Human Faces

arXiv: https://arxiv.org/abs/2204.06607 · HTML: https://mlsanity.com/paper/2204.06607/

## Key facts

- Cited by 165 papers in the map (data through 7 Oct 2026).
- Builds on 58 of its 93 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2012.00926/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Neural Voice Puppetry: Audio-driven Facial Reenactment](https://mlsanity.com/paper/1912.05566/) (2019)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [Neural Head Avatars from Monocular RGB Videos](https://mlsanity.com/paper/2112.01554/) (2021)
- [FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction](https://mlsanity.com/paper/2003.13989/) (2020)
- [Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network](https://mlsanity.com/paper/1803.07835/) (2018)
- [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz](https://mlsanity.com/paper/1712.02859/) (2017)

## Built on by (most cited first, 30 of 165)

- [A Survey on Video Diffusion Models](https://mlsanity.com/paper/2310.10647/) (2023)
- [Generating Holistic 3D Human Motion from Speech](https://mlsanity.com/paper/2212.04420/) (2022)
- [Instant Volumetric Head Avatars](https://mlsanity.com/paper/2211.12499/) (2022)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Rendering](https://mlsanity.com/paper/2312.11360/) (2023)
- [DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models](https://mlsanity.com/paper/2310.00434/) (2023)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [Imitator: Personalized Speech-driven 3D Facial Animation](https://mlsanity.com/paper/2301.00023/) (2022)
- [3D Gaussian Blendshapes for Head Avatar Animation](https://mlsanity.com/paper/2404.19398/) (2024)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [Avat3r: Large Animatable Gaussian Reconstruction Model for High-fidelity 3D Head Avatars](https://mlsanity.com/paper/2502.20220/) (2025)
- [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022)
- [HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors](https://mlsanity.com/paper/2408.06019/) (2024)
- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [Text-Guided Generation and Editing of Compositional 3D Avatars](https://mlsanity.com/paper/2309.07125/) (2023)
- [FLARE: Fast Learning of Animatable and Relightable Mesh Avatars](https://mlsanity.com/paper/2310.17519/) (2023)
- [3D Face Reconstruction with the Geometric Guidance of Facial Part Segmentation](https://mlsanity.com/paper/2312.00311/) (2023)
- [MonoNPHM: Dynamic Head Reconstruction from Monocular Videos](https://mlsanity.com/paper/2312.06740/) (2023)
- [Instant Multi-View Head Capture through Learnable Registration](https://mlsanity.com/paper/2306.07437/) (2023)
- [HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting](https://mlsanity.com/paper/2402.06149/) (2024)
- [FATE: Full-head Gaussian Avatar with Textural Editing from Monocular Video](https://mlsanity.com/paper/2411.15604/) (2024)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [ARTalk: Speech-Driven 3D Head Animation via Autoregressive Model](https://mlsanity.com/paper/2502.20323/) (2025)
- [LaughTalk: Expressive 3D Talking Head Generation with Laughter](https://mlsanity.com/paper/2311.00994/) (2023)
- [RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars](https://mlsanity.com/paper/2503.12886/) (2025)
- [HRAvatar: High-Quality and Relightable Gaussian Head Avatar](https://mlsanity.com/paper/2503.08224/) (2025)
- [Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation](https://mlsanity.com/paper/2401.04728/) (2024)
- [Look Ma, no markers: holistic performance capture without the hassle](https://mlsanity.com/paper/2410.11520/) (2024)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)

## Abstract (arXiv)

Face reconstruction and tracking is a building block of numerous applications in AR/VR, human-machine interaction, as well as medical applications. Most of these applications rely on a metrically correct prediction of the shape, especially, when the reconstructed subject is put into a metrical context (i.e., when there is a reference object of known size). A metrical reconstruction is also needed for any application that measures distances and dimensions of the subject (e.g., to virtually fit a glasses frame). State-of-the-art methods for face reconstruction from a single image are trained on large 2D image datasets in a self-supervised fashion. However, due to the nature of a perspective projection they are not able to reconstruct the actual face dimensions, and even predicting the average human face outperforms some of these methods in a metrical sense. To learn the actual shape of a face, we argue for a supervised training scheme. Since there exists no large-scale 3D dataset for this task, we annotated and unified small- and medium-scale databases. The resulting unified dataset is still a medium-scale dataset with more than 2k identities and training purely on it would lead to overfitting. To this end, we take advantage of a face recognition network pretrained on a large-scale 2D image dataset, which provides distinct features for different faces and is robust to expression, illumination, and camera changes. Using these features, we train our face shape estimator in a supervised fashion, inheriting the robustness and generalization of the face recognition network. Our method, which we call MICA (MetrIC fAce), outperforms the state-of-the-art reconstruction methods by a large margin, both on current non-metric benchmarks as well as on our metric benchmarks (15% and 24% lower average error on NoW, respectively).