# Self-supervised Learning of Detailed 3D Face Reconstruction

arXiv: https://arxiv.org/abs/1910.11791 · HTML: https://mlsanity.com/paper/1910.11791/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2019.

## Built on by (most cited first, 21 of 21)

- [FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset](https://mlsanity.com/paper/2203.14057/) (2022)
- [Model-based 3D Hand Reconstruction via Self-Supervised Learning](https://mlsanity.com/paper/2103.11703/) (2021)
- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)
- [REALY: Rethinking the Evaluation of 3D Face Reconstruction](https://mlsanity.com/paper/2203.09729/) (2022)
- [DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation](https://mlsanity.com/paper/2203.07931/) (2022)
- [Consistent 3D Hand Reconstruction in Video via self-supervised Learning](https://mlsanity.com/paper/2201.09548/) (2022)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details](https://mlsanity.com/paper/2303.11225/) (2023)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)
- [Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape](https://mlsanity.com/paper/2204.04379/) (2022)
- [Structure-aware Editable Morphable Model for 3D Facial Detail Animation and Manipulation](https://mlsanity.com/paper/2207.09019/) (2022)
- [Light Sampling Field and BRDF Representation for Physically-based Neural Rendering](https://mlsanity.com/paper/2304.05472/) (2023)
- [Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images](https://mlsanity.com/paper/2303.10896/) (2023)
- [FLAME-based Multi-View 3D Face Reconstruction](https://mlsanity.com/paper/2308.07551/) (2023)
- [Sphere Face Model:A 3D Morphable Model with Hypersphere Manifold Latent Space](https://mlsanity.com/paper/2112.02238/) (2021)
- [BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs](https://mlsanity.com/paper/2203.05908/) (2022)
- [Self-supervised Re-renderable Facial Albedo Reconstruction from Single Image](https://mlsanity.com/paper/2111.08282/) (2021)
- [From 2D Images to 3D Model:Weakly Supervised Multi-View Face Reconstruction with Deep Fusion](https://mlsanity.com/paper/2204.03842/) (2022)
- [Fake It Without Making It: Conditioned Face Generation for Accurate 3D Face Reconstruction](https://mlsanity.com/paper/2307.13639/) (2023)
- [Hierarchical MLANet: Multi-level Attention for 3D Face Reconstruction From Single Images](https://mlsanity.com/paper/2509.10024/) (2025)

## Abstract (arXiv)

In this paper, we present an end-to-end learning framework for detailed 3D face reconstruction from a single image. Our approach uses a 3DMM-based coarse model and a displacement map in UV-space to represent a 3D face. Unlike previous work addressing the problem, our learning framework does not require supervision of surrogate ground-truth 3D models computed with traditional approaches. Instead, we utilize the input image itself as supervision during learning. In the first stage, we combine a photometric loss and a facial perceptual loss between the input face and the rendered face, to regress a 3DMM-based coarse model. In the second stage, both the input image and the regressed texture of the coarse model are unwrapped into UV-space, and then sent through an image-toimage translation network to predict a displacement map in UVspace. The displacement map and the coarse model are used to render a final detailed face, which again can be compared with the original input image to serve as a photometric loss for the second stage. The advantage of learning displacement map in UV-space is that face alignment can be explicitly done during the unwrapping, thus facial details are easier to learn from large amount of data. Extensive experiments demonstrate the superiority of the proposed method over previous work.