mlsanityData to 14 Oct 2026

Learning to regulate 3D head shape by removing occluding hair from in-the-wild images

arXiv 2208.12078 ↗First posted 25 Aug 2022cs.CV cs.GRcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent 3D face reconstruction methods reconstruct the entire head compared to earlier approaches which only model the face. Although these methods accurately reconstruct facial features, they do not explicitly regulate the upper part of the head. Extracting information about this part of the head is challenging due to varying degrees of occlusion by hair. We present a novel approach for modeling the upper head by removing occluding hair and reconstructing the skin, revealing information about the head shape. We introduce three objectives: 1) a dice consistency loss that enforces similarity between the overall head shape of the source and rendered image, 2) a scale consistency loss to ensure that head shape is accurately reproduced even if the upper part of the head is not visible, and 3) a 71 landmark detector trained using a moving average loss function to detect additional landmarks on the head. These objectives are used to train an encoder in an unsupervised manner to regress FLAME parameters from in-the-wild input images. Our unsupervised 3DMM model achieves state-of-the-art results on popular benchmarks and can be used to infer the head shape, facial features, and textures for direct use in animation or avatar creation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 41 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

29 of 41 references.

References · 29

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  7. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  8. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 914
  9. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  10. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  11. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 632
  12. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
17 more
  1. 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 525
  2. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  3. 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 482
  4. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  5. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  6. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 394
  7. 2021GAN Inversion: A Surveycited by 367
  8. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  9. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  10. 2018Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Networkcited by 195
  11. 2019Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervisioncited by 176
  12. 2019EdgeConnect: Generative Image Inpainting with Adversarial Edge Learningcited by 163
  13. 2019GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstructioncited by 150
  14. 2016Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Networkcited by 109
  15. 2018Unsupervised Training for 3D Morphable Model Regressioncited by 100
  16. 2020Adaptive Graph Convolutional Network with Attention Graph Clustering for Co-saliency Detectioncited by 35
  17. 2020Hyperrealistic Image Inpainting with Hypergraphscited by 6

Built on by

Not cited yet.