# Face De-occlusion using 3D Morphable Model and Generative Adversarial Network

arXiv: https://arxiv.org/abs/1904.06109 · HTML: https://mlsanity.com/paper/1904.06109/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Apr 2019.

## Built on by (most cited first, 11 of 11)

- [Facial Geometric Detail Recovery via Implicit Representation](https://mlsanity.com/paper/2203.09692/) (2022)
- [Deep Face Video Inpainting via UV Mapping](https://mlsanity.com/paper/2109.00681/) (2021)
- [Segmentation-Reconstruction-Guided Facial Image De-occlusion](https://mlsanity.com/paper/2112.08022/) (2021)
- [Contrastive Attention Network with Dense Field Estimation for Face Completion](https://mlsanity.com/paper/2112.10310/) (2021)
- [Locality-aware Channel-wise Dropout for Occluded Face Recognition](https://mlsanity.com/paper/2107.09270/) (2021)
- [Indian Masked Faces in the Wild Dataset](https://mlsanity.com/paper/2106.09670/) (2021)
- [Self-Supervised Robustifying Guidance for Monocular 3D Face Reconstruction](https://mlsanity.com/paper/2112.14382/) (2021)
- [3DFaceFill: An Analysis-By-Synthesis Approach to Face Completion](https://mlsanity.com/paper/2110.10395/) (2021)
- [BIFRNet: A Brain-Inspired Feature Restoration DNN for Partially Occluded Image Recognition](https://mlsanity.com/paper/2303.01309/) (2023)
- [Delta-GAN-Encoder: Encoding Semantic Changes for Explicit Image Editing, using Few Synthetic Samples](https://mlsanity.com/paper/2111.08419/) (2021)
- [Surface Geometry Processing: An Efficient Normal-based Detail Representation](https://mlsanity.com/paper/2307.07945/) (2023)

## Abstract (arXiv)

In recent decades, 3D morphable model (3DMM) has been commonly used in image-based photorealistic 3D face reconstruction. However, face images are often corrupted by serious occlusion by non-face objects including eyeglasses, masks, and hands. Such objects block the correct capture of landmarks and shading information. Therefore, the reconstructed 3D face model is hardly reusable. In this paper, a novel method is proposed to restore de-occluded face images based on inverse use of 3DMM and generative adversarial network. We utilize the 3DMM prior to the proposed adversarial network and combine a global and local adversarial convolutional neural network to learn face de-occlusion model. The 3DMM serves not only as geometric prior but also proposes the face region for the local discriminator. Experiment results confirm the effectiveness and robustness of the proposed algorithm in removing challenging types of occlusions with various head poses and illumination. Furthermore, the proposed method reconstructs the correct 3D face model with de-occluded textures.