mlsanityData to 7 Oct 2026

Generating Diverse 3D Reconstructions from a Single Occluded Face Image

arXiv 2112.00879 ↗First posted 1 Dec 2021cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Occlusions are a common occurrence in unconstrained face images. Single image 3D reconstruction from such face images often suffers from corruption due to the presence of occlusions. Furthermore, while a plurality of 3D reconstructions is plausible in the occluded regions, existing approaches are limited to generating only a single solution. To address both of these challenges, we present Diverse3DFace, which is specifically designed to simultaneously generate a diverse and realistic set of 3D reconstructions from a single occluded face image. It consists of three components: a global+local shape fitting process, a graph neural network-based mesh VAE, and a Determinantal Point Process based diversity promoting iterative optimization procedure. Quantitative and qualitative comparisons of 3D reconstruction on occluded faces show that Diverse3DFace can estimate 3D shapes that are consistent with the visible regions in the target image while exhibiting high, yet realistic, levels of diversity on the occluded regions. On face images occluded by masks, glasses, and other random objects, Diverse3DFace generates a distribution of 3D shapes having ~50% higher diversity on the occluded regions compared to the baselines. Moreover, our closest sample to the ground truth has ~40% lower MSE than the singular reconstructions by existing approaches.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 38 of its 63 references that are in the map. First posted on arXiv 1 Dec 2021.

Builds on

38 of 63 references.

References · 38

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2017Graph Attention Networkscited by 8,854
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  6. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  7. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  8. 2018Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networkscited by 878
  9. 20193D Morphable Face Models -- Past, Present and Futurecited by 269
  10. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 267
  11. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 199
  12. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
26 more
  1. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  2. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  3. 2017MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstructioncited by 163
  4. 2019Pluralistic Image Completioncited by 162
  5. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  6. 2017Morphable Face Models - An Open Frameworkcited by 138
  7. 2016Mode Regularized Generative Adversarial Networkscited by 123
  8. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
  9. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  10. 2020i3DMM: Deep Implicit 3D Morphable Model of Human Headscited by 109
  11. 2019Towards High-fidelity Nonlinear 3D Face Morphable Modelcited by 91
  12. 2019Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generationcited by 89
  13. 2016Joint Multimodal Learning with Deep Generative Modelscited by 87
  14. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
  15. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  16. 2019Diversity-Sensitive Conditional Generative Adversarial Networkscited by 77
  17. 2017On Face Segmentation, Face Swapping, and Face Perceptioncited by 71
  18. 2017Neural Face Editing with Intrinsic Image Disentanglingcited by 70
  19. 2019Towards a complete 3D morphable model of the human headcited by 63
  20. 2017Multi-Agent Diverse Generative Adversarial Networkscited by 59
  21. 2018On Learning 3D Face Morphable Model from In-the-wild Imagescited by 54
  22. 2020Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernelscited by 53
  23. 2019Explorable Super Resolutioncited by 44
  24. 2018GDPP: Learning Diverse Generations Using Determinantal Point Processcited by 26
  25. 20193D Dense Face Alignment via Graph Convolution Networkscited by 14
  26. 2016Real-Time Facial Segmentation and Performance Capture from RGB Inputcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3Dcited by 27
  2. 2023Text-Guided 3D Face Synthesis -- From Generation to Editingcited by 8
  3. 2024OFER: Occluded Face Expression Reconstructioncited by 0
  4. 2026RAGMesh with FaME-G2E: Long-Form Text-Driven 3D Face Generation and Editingcited by 0