# InverseFaceNet: Deep Monocular Inverse Face Rendering

arXiv: https://arxiv.org/abs/1703.10956 · HTML: https://mlsanity.com/paper/1703.10956/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2017.

## Built on by (most cited first, 6 of 6)

- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation](https://mlsanity.com/paper/2103.15876/) (2021)
- [DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs](https://mlsanity.com/paper/2302.12253/) (2023)
- [A Perceptual Shape Loss for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2310.19580/) (2023)
- [GaNI: Global and Near Field Illumination Aware Neural Inverse Rendering](https://mlsanity.com/paper/2403.15651/) (2024)
- [Generating Animatable 3D Cartoon Faces from Single Portraits](https://mlsanity.com/paper/2307.01468/) (2023)

## Abstract (arXiv)

We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image. By estimating all parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible in real time. Most previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created training corpus. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. We further propose a self-supervised bootstrapping process in the network training loop, which iteratively updates the synthetic training corpus to better reflect the distribution of real-world imagery. We demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.