# Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

arXiv: https://arxiv.org/abs/2105.07474 · HTML: https://mlsanity.com/paper/2105.07474/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 61 references that are in the map.
- First posted on arXiv 16 May 2021.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [NVAE: A Deep Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2007.03898/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [NIPS 2016 Tutorial: Generative Adversarial Networks](https://mlsanity.com/paper/1701.00160/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision](https://mlsanity.com/paper/1905.06817/) (2019)
- [Deep Appearance Models for Face Rendering](https://mlsanity.com/paper/1808.00362/) (2018)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1804.03786/) (2018)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network](https://mlsanity.com/paper/1612.04904/) (2016)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hz](https://mlsanity.com/paper/1712.02859/) (2017)
- [Towards High-fidelity Nonlinear 3D Face Morphable Model](https://mlsanity.com/paper/1904.04933/) (2019)
- [AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"](https://mlsanity.com/paper/2003.13845/) (2020)
- [FML: Face Model Learning from Videos](https://mlsanity.com/paper/1812.07603/) (2018)
- [Learning Detailed Face Reconstruction from a Single Image](https://mlsanity.com/paper/1611.05053/) (2016)
- [Neural Face Editing with Intrinsic Image Disentangling](https://mlsanity.com/paper/1704.04131/) (2017)
- [3D Face Reconstruction by Learning from Synthetic Data](https://mlsanity.com/paper/1609.04387/) (2016)
- [Towards a complete 3D morphable model of the human head](https://mlsanity.com/paper/1911.08008/) (2019)
- [UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition](https://mlsanity.com/paper/1712.04695/) (2017)
- [Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation](https://mlsanity.com/paper/1703.10131/) (2017)
- [On Learning 3D Face Morphable Model from In-the-wild Images](https://mlsanity.com/paper/1808.09560/) (2018)
- [A Morphable Face Albedo Model](https://mlsanity.com/paper/2004.02711/) (2020)
- [3D Face Morphable Models "In-the-Wild"](https://mlsanity.com/paper/1701.05360/) (2017)
- [BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography](https://mlsanity.com/paper/1704.08614/) (2017)
- [OSTeC: One-Shot Texture Completion](https://mlsanity.com/paper/2012.15370/) (2020)

## Built on by (most cited first, 30 of 33)

- [Towards Metrical Reconstruction of Human Faces](https://mlsanity.com/paper/2204.06607/) (2022)
- [ClipFace: Text-guided Editing of Textured 3D Morphable Models](https://mlsanity.com/paper/2212.01406/) (2022)
- [A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2302.14434/) (2023)
- [FitMe: Deep Photorealistic 3D Morphable Model Avatars](https://mlsanity.com/paper/2305.09641/) (2023)
- [HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field](https://mlsanity.com/paper/2309.17128/) (2023)
- [Relightify: Relightable 3D Faces from a Single Image via Diffusion Models](https://mlsanity.com/paper/2305.06077/) (2023)
- [REALY: Rethinking the Evaluation of 3D Face Reconstruction](https://mlsanity.com/paper/2203.09729/) (2022)
- [VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment](https://mlsanity.com/paper/2312.04651/) (2023)
- [AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars](https://mlsanity.com/paper/2306.00547/) (2023)
- [Towards Realistic Generative 3D Face Models](https://mlsanity.com/paper/2304.12483/) (2023)
- [3DMM-RF: Convolutional Radiance Fields for 3D Face Modeling](https://mlsanity.com/paper/2209.07366/) (2022)
- [Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstruction](https://mlsanity.com/paper/2310.08784/) (2023)
- [Unsupervised High-Fidelity Facial Texture Generation and Reconstruction](https://mlsanity.com/paper/2110.04760/) (2021)
- [Learning a Generalized Physical Face Model From Data](https://mlsanity.com/paper/2402.19477/) (2024)
- [PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editing](https://mlsanity.com/paper/2511.02777/) (2025)
- [FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Models](https://mlsanity.com/paper/2312.04465/) (2023)
- [Geometry-aware Two-scale PIFu Representation for Human Reconstruction](https://mlsanity.com/paper/2112.02082/) (2021)
- [Self-Supervised Robustifying Guidance for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2112.14382/) (2021)
- [Evaluating Deepfake Detectors in the Wild](https://mlsanity.com/paper/2507.21905/) (2025)
- [Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterization](https://mlsanity.com/paper/2204.01386/) (2022)
- [High-Fidelity Facial Albedo Estimation via Texture Quantization](https://mlsanity.com/paper/2406.13149/) (2024)
- [FAGhead: Fully Animate Gaussian Head from Monocular Videos](https://mlsanity.com/paper/2406.19070/) (2024)
- [Tex-ViT: A Generalizable, Robust, Texture-based dual-branch cross-attention deepfake detector](https://mlsanity.com/paper/2408.16892/) (2024)
- [REFA: Real-time Egocentric Facial Animations for Virtual Reality](https://mlsanity.com/paper/2601.03507/) (2026)
- [Semantic RGB-D Image Synthesis](https://mlsanity.com/paper/2308.11356/) (2023)
- [3D Face Reconstruction Error Decomposed: A Modular Benchmark for Fair and Fast Method Evaluation](https://mlsanity.com/paper/2505.18025/) (2025)
- [FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Image](https://mlsanity.com/paper/2606.24232/) (2026)
- [Domain Engineering for Applied Monocular Reconstruction of Parametric Faces](https://mlsanity.com/paper/2209.02600/) (2022)
- [Controllable Face Manipulation and UV Map Generation by Self-supervised Learning](https://mlsanity.com/paper/2209.12050/) (2022)
- [Efficient Hair Style Transfer with Generative Adversarial Networks](https://mlsanity.com/paper/2210.12524/) (2022)

## Abstract (arXiv)

A lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the recent works, the texture features either correspond to components of a linear texture space or are learned by auto-encoders directly from in-the-wild images. In all cases, the quality of the facial texture reconstruction is still not capable of modeling facial texture with high-frequency details. In this paper, we take a radically different approach and harness the power of Generative Adversarial Networks (GANs) and DCNNs in order to reconstruct the facial texture and shape from single images. That is, we utilize GANs to train a very powerful facial texture prior editfrom a large-scale 3D texture dataset. Then, we revisit the original 3D Morphable Models (3DMMs) fitting making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. In order to be robust towards initialisation and expedite the fitting process, we propose a novel self-supervised regression based approach. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details.