mlsanityData to 7 Oct 2026

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

arXiv 2105.07474 ↗First posted 16 May 2021cs.CVcited by 33

Not on the map. Why?

Abstract (arXiv)

A lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the recent works, the texture features either correspond to components of a linear texture space or are learned by auto-encoders directly from in-the-wild images. In all cases, the quality of the facial texture reconstruction is still not capable of modeling facial texture with high-frequency details. In this paper, we take a radically different approach and harness the power of Generative Adversarial Networks (GANs) and DCNNs in order to reconstruct the facial texture and shape from single images. That is, we utilize GANs to train a very powerful facial texture prior editfrom a large-scale 3D texture dataset. Then, we revisit the original 3D Morphable Models (3DMMs) fitting making use of non-linear optimization to find the optimal latent parameters that best reconstruct the test image but under a new perspective. In order to be robust towards initialisation and expedite the fitting process, we propose a novel self-supervised regression based approach. We demonstrate excellent results in photorealistic and identity preserving 3D face reconstructions and achieve for the first time, to the best of our knowledge, facial texture reconstruction with high-frequency details.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 16 May 2021.

Builds on

38 of 61 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  4. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  5. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  6. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  7. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  8. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  9. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  10. 2018Deep Appearance Models for Face Renderingcited by 164
  11. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  12. 2018Nonlinear 3D Face Morphable Modelcited by 157
26 more
  1. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  2. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  3. 2018Unsupervised Training for 3D Morphable Model Regressioncited by 99
  4. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  5. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  6. 2020AvatarMe: Realistically Renderable 3D Facial Reconstruction "in-the-wild"cited by 91
  7. 2018FML: Face Model Learning from Videoscited by 78
  8. 2016Learning Detailed Face Reconstruction from a Single Imagecited by 71
  9. 2017Neural Face Editing with Intrinsic Image Disentanglingcited by 70
  10. 20163D Face Reconstruction by Learning from Synthetic Datacited by 64
  11. 2019Towards a complete 3D morphable model of the human headcited by 63
  12. 2017UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognitioncited by 62
  13. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  14. 2018On Learning 3D Face Morphable Model from In-the-wild Imagescited by 54
  15. 2020A Morphable Face Albedo Modelcited by 54
  16. 20173D Face Morphable Models "In-the-Wild"cited by 44
  17. 2017BAM! The Behance Artistic Media Dataset for Recognition Beyond Photographycited by 43
  18. 2020OSTeC: One-Shot Texture Completioncited by 37
  19. 2019Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networkscited by 34
  20. 2017Synthesizing Normalized Faces from Facial Identity Featurescited by 33
  21. 2016Photorealistic Facial Texture Inference Using Deep Neural Networkscited by 32
  22. 2018Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Modelcited by 28
  23. 20173D Face Reconstruction with Geometry Details from a Single Imagecited by 27
  24. 2018High Quality Facial Surface and Texture Synthesis via Generative Adversarial Networkscited by 22
  25. 2016Fitting a 3D Morphable Model to Edges: A Comparison Between Hard and Soft Correspondencescited by 17
  26. 20174DFAB: A Large Scale 4D Facial Expression Database for Biometric Applicationscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 8; 2023: 11; 2024: 4; 2025: 3; 2026: 4.

33 papers.

Sort:

By area:

  1. 2022Towards Metrical Reconstruction of Human Facescited by 165
  2. 2022ClipFace: Text-guided Editing of Textured 3D Morphable Modelscited by 59
  3. 2023A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Imagescited by 43
  4. 2023FitMe: Deep Photorealistic 3D Morphable Model Avatarscited by 38
  5. 2023HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Fieldcited by 37
  6. 2023Relightify: Relightable 3D Faces from a Single Image via Diffusion Modelscited by 34
  7. 2023VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactmentcited by 33
  8. 2022REALY: Rethinking the Evaluation of 3D Face Reconstructioncited by 32
  9. 2023AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatarscited by 29
  10. 2023Towards Realistic Generative 3D Face Modelscited by 20
  11. 20223DMM-RF: Convolutional Radiance Fields for 3D Face Modelingcited by 10
  12. 2023Implicit Shape and Appearance Priors for Few-Shot Full Head Reconstructioncited by 8
21 more
  1. 2021Unsupervised High-Fidelity Facial Texture Generation and Reconstructioncited by 6
  2. 2024Learning a Generalized Physical Face Model From Datacited by 6
  3. 2025PercHead: Perceptual Head Model for Single-Image 3D Head Reconstruction & Editingcited by 6
  4. 2023FitDiff: Robust monocular 3D facial shape and reflectance estimation using Diffusion Modelscited by 5
  5. 2021Geometry-aware Two-scale PIFu Representation for Human Reconstructioncited by 4
  6. 2021Self-Supervised Robustifying Guidance for Monocular 3D Face Reconstructioncited by 3
  7. 2025Evaluating Deepfake Detectors in the Wildcited by 3
  8. 2022Dressi: A Hardware-Agnostic Differentiable Renderer with Reactive Shader Packing and Soft Rasterizationcited by 2
  9. 2024High-Fidelity Facial Albedo Estimation via Texture Quantizationcited by 2
  10. 2024FAGhead: Fully Animate Gaussian Head from Monocular Videoscited by 2
  11. 2024Tex-ViT: A Generalizable, Robust, Texture-based dual-branch cross-attention deepfake detectorcited by 2
  12. 2023Semantic RGB-D Image Synthesiscited by 1
  13. 20253D Face Reconstruction Error Decomposed: A Modular Benchmark for Fair and Fast Method Evaluationcited by 1
  14. 2026REFA: Real-time Egocentric Facial Animations for Virtual Realitycited by 1
  15. 2022Domain Engineering for Applied Monocular Reconstruction of Parametric Facescited by 0
  16. 2022Controllable Face Manipulation and UV Map Generation by Self-supervised Learningcited by 0
  17. 2022Efficient Hair Style Transfer with Generative Adversarial Networkscited by 0
  18. 2023MUNCH: Modelling Unique 'N Controllable Headscited by 0
  19. 2026Dual Diffusion Models for Multi-modal Guided 3D Avatar Generationcited by 0
  20. 2026Revisiting an Old Perspective Projection for Monocular 3D Morphable Models Regressioncited by 0
  21. 2026FiCA: Feed-forward instant Gaussian Codec Avatars from a Single Portrait Imagecited by 0