# Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation

arXiv: https://arxiv.org/abs/1703.10131 · HTML: https://mlsanity.com/paper/1703.10131/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2017.

## Built on by (most cited first, 30 of 56)

- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [Fake It Till You Make It: Face analysis in the wild using synthetic data alone](https://mlsanity.com/paper/2109.15102/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset](https://mlsanity.com/paper/2203.14057/) (2022)
- [H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction](https://mlsanity.com/paper/2107.12512/) (2021)
- [3D face reconstruction with dense landmarks](https://mlsanity.com/paper/2204.02776/) (2022)
- [AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs](https://mlsanity.com/paper/2112.05957/) (2021)
- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction](https://mlsanity.com/paper/2111.01082/) (2021)
- [Text and Image Guided 3D Avatar Generation and Manipulation](https://mlsanity.com/paper/2202.06079/) (2022)
- [Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/2105.07474/) (2021)
- [REALY: Rethinking the Evaluation of 3D Face Reconstruction](https://mlsanity.com/paper/2203.09729/) (2022)
- [Riggable 3D Face Reconstruction via In-Network Optimization](https://mlsanity.com/paper/2104.03493/) (2021)
- [SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction](https://mlsanity.com/paper/2106.03021/) (2021)
- [Facial Geometric Detail Recovery via Implicit Representation](https://mlsanity.com/paper/2203.09692/) (2022)
- [Self-Supervised 3D Face Reconstruction via Conditional Estimation](https://mlsanity.com/paper/2110.04800/) (2021)
- [S2F2: Self-Supervised High Fidelity Face Reconstruction from Monocular Image](https://mlsanity.com/paper/2203.07732/) (2022)
- [3D-FM GAN: Towards 3D-Controllable Face Manipulation](https://mlsanity.com/paper/2208.11257/) (2022)
- [3DCaricShop: A Dataset and A Baseline Method for Single-view 3D Caricature Face Reconstruction](https://mlsanity.com/paper/2103.08204/) (2021)
- [Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection](https://mlsanity.com/paper/2106.07852/) (2021)
- [High-fidelity Face Tracking for AR/VR via Deep Lighting Adaptation](https://mlsanity.com/paper/2103.15876/) (2021)
- [SIRA: Relightable Avatars from a Single Image](https://mlsanity.com/paper/2209.03027/) (2022)
- [BareSkinNet: De-makeup and De-lighting via 3D Face Reconstruction](https://mlsanity.com/paper/2209.09029/) (2022)
- [SIDER: Single-Image Neural Optimization for Facial Geometric Detail Recovery](https://mlsanity.com/paper/2108.05465/) (2021)
- [Controllable 3D Generative Adversarial Face Model via Disentangling Shape and Appearance](https://mlsanity.com/paper/2208.14263/) (2022)
- [Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction](https://mlsanity.com/paper/2310.08784/) (2023)
- [Mesh Density Adaptation for Template-based Shape Reconstruction](https://mlsanity.com/paper/2307.16205/) (2023)
- [Unsupervised High-Fidelity Facial Texture Generation and Reconstruction](https://mlsanity.com/paper/2110.04760/) (2021)
- [Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions](https://mlsanity.com/paper/2209.04436/) (2022)
- [RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs](https://mlsanity.com/paper/2302.05486/) (2023)

## Abstract (arXiv)

It has been recently shown that neural networks can recover the geometric structure of a face from a single given image. A common denominator of most existing face geometry reconstruction methods is the restriction of the solution space to some low-dimensional subspace. While such a model significantly simplifies the reconstruction problem, it is inherently limited in its expressiveness. As an alternative, we propose an Image-to-Image translation network that jointly maps the input image to a depth image and a facial correspondence map. This explicit pixel-based mapping can then be utilized to provide high quality reconstructions of diverse faces under extreme expressions, using a purely geometric refinement process. In the spirit of recent approaches, the network is trained only with synthetic data, and is then evaluated on in-the-wild facial images. Both qualitative and quantitative analyses demonstrate the accuracy and the robustness of our approach.