# Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision

arXiv: https://arxiv.org/abs/1905.06817 · HTML: https://mlsanity.com/paper/1905.06817/

## Key facts

- Cited by 176 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2019.

## Built on by (most cited first, 30 of 176)

- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [Realistic One-shot Mesh-based Head Avatars](https://mlsanity.com/paper/2206.08343/) (2022)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Generative Neural Articulated Radiance Fields](https://mlsanity.com/paper/2206.14314/) (2022)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping](https://mlsanity.com/paper/2106.09965/) (2021)
- [GAN-Control: Explicitly Controllable GANs](https://mlsanity.com/paper/2101.02477/) (2021)
- [SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesis](https://mlsanity.com/paper/2404.04104/) (2024)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [DiffFace: Diffusion-based Face Swapping with Facial Guidance](https://mlsanity.com/paper/2212.13344/) (2022)
- [DiffusionRig: Learning Personalized Priors for Facial Appearance Editing](https://mlsanity.com/paper/2304.06711/) (2023)
- [A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis](https://mlsanity.com/paper/2110.15678/) (2021)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting](https://mlsanity.com/paper/2402.01459/) (2024)
- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [NOFA: NeRF-based One-shot Facial Avatar Reconstruction](https://mlsanity.com/paper/2307.03441/) (2023)
- [BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Information](https://mlsanity.com/paper/2203.15536/) (2022)
- [Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objects](https://mlsanity.com/paper/2112.11347/) (2021)
- [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022)
- [HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery](https://mlsanity.com/paper/2304.05690/) (2023)
- [Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing](https://mlsanity.com/paper/2103.15432/) (2021)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [FitMe: Deep Photorealistic 3D Morphable Model Avatars](https://mlsanity.com/paper/2305.09641/) (2023)

## Abstract (arXiv)

The estimation of 3D face shape from a single image must be robust to variations in lighting, head pose, expression, facial hair, makeup, and occlusions. Robustness requires a large training set of in-the-wild images, which by construction, lack ground truth 3D shape. To train a network without any 2D-to-3D supervision, we present RingNet, which learns to compute 3D face shape from a single image. Our key observation is that an individual's face shape is constant across images, regardless of expression, pose, lighting, etc. RingNet leverages multiple images of a person and automatically detected 2D face features. It uses a novel loss that encourages the face shape to be similar when the identity is the same and different for different people. We achieve invariance to expression by representing the face using the FLAME model. Once trained, our method takes a single image and outputs the parameters of FLAME, which can be readily animated. Additionally we create a new database of faces `not quite in-the-wild' (NoW) with 3D head scans and high-resolution images of the subjects in a wide variety of conditions. We evaluate publicly available methods and find that RingNet is more accurate than methods that use 3D supervision. The dataset, model, and results are available for research purposes at http://ringnet.is.tuebingen.mpg.de.