mlsanityData to 7 Oct 2026

Fake It Without Making It: Conditioned Face Generation for Accurate 3D Face Reconstruction

arXiv 2307.13639 ↗First posted 25 Jul 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate 3D face reconstruction from 2D images is an enabling technology with applications in healthcare, security, and creative industries. However, current state-of-the-art methods either rely on supervised training with very limited 3D data or self-supervised training with 2D image data. To bridge this gap, we present a method to generate a large-scale synthesised dataset of 250K photorealistic images and their corresponding shape parameters and depth maps, which we call SynthFace. Our synthesis method conditions Stable Diffusion on depth maps sampled from the FLAME 3D Morphable Model (3DMM) of the human face, allowing us to generate a diverse set of shape-consistent facial images that is designed to be balanced in race and gender. We further propose ControlFace, a deep neural network, trained on SynthFace, which achieves competitive performance on the NoW benchmark, without requiring 3D supervision or manual 3D asset creation. The complete SynthFace dataset will be made publicly available upon publication.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

33 of 38 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  11. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  12. 2016HyperNetworkscited by 1,003
21 more
  1. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
  2. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  3. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  4. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  5. 2021Fake It Till You Make It: Face analysis in the wild using synthetic data alonecited by 200
  6. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  7. 2022Towards Metrical Reconstruction of Human Facescited by 165
  8. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  9. 2019Meta-Sim: Learning to Generate Synthetic Datasetscited by 128
  10. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  11. 2017Self-supervised Multi-level Face Model Learning for Monocular Reconstruction at over 250 Hzcited by 91
  12. 2020Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistencycited by 78
  13. 20223D face reconstruction with dense landmarkscited by 70
  14. 20163D Face Reconstruction by Learning from Synthetic Datacited by 64
  15. 2017Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translationcited by 56
  16. 2020GIF: Generative Interpretable Facescited by 53
  17. 2019MVF-Net: Multi-View 3D Face Morphable Model Regressioncited by 45
  18. 2022Controllable 3D Face Synthesis with Conditional Generative Occupancy Fieldscited by 37
  19. 2018Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognitioncited by 27
  20. 2019Self-supervised Learning of Detailed 3D Face Reconstructioncited by 21
  21. 2023Towards Realistic Generative 3D Face Modelscited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Bringing Diversity from Diffusion Models to Semantic-Guided Face Asset Generation
  2. 2025Data Synthesis with Diverse Styles for Face Recognition via 3DMM-Guided Diffusion
  3. 2024FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads
  4. 2023DCFace: Synthetic Face Generation with Dual Condition Diffusion Model
  5. 2026Identity-Conditioned Latent Consistency Distillation for Face Synthesis
  6. 2024TCDiff: Triple Condition Diffusion Model with 3D Constraints for Stylizing Synthetic Faces
  7. 2024OFER: Occluded Face Expression Reconstruction
  8. 2023Text-Guided 3D Face Synthesis -- From Generation to Editing