# Learning Representations for Masked Facial Recovery

arXiv: https://arxiv.org/abs/2212.14110 · HTML: https://mlsanity.com/paper/2212.14110/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 28 Dec 2022.

## Builds on (most cited first, 30 of 30)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [Adversarial Latent Autoencoders](https://mlsanity.com/paper/2004.04467/) (2020)
- [Foreground-aware Image Inpainting](https://mlsanity.com/paper/1901.05945/) (2019)
- [Improved StyleGAN Embedding: Where are the Good Latents?](https://mlsanity.com/paper/2012.09036/) (2020)
- [Generative Face Completion](https://mlsanity.com/paper/1704.05838/) (2017)
- [Intermediate Layer Optimization for Inverse Problems using Deep Generative Models](https://mlsanity.com/paper/2102.07364/) (2021)
- [Masked Face Recognition Dataset and Application](https://mlsanity.com/paper/2003.09093/) (2020)
- [Labels4Free: Unsupervised Segmentation using StyleGAN](https://mlsanity.com/paper/2103.14968/) (2021)
- [Masked Face Recognition for Secure Authentication](https://mlsanity.com/paper/2008.11104/) (2020)
- [A survey of face recognition techniques under occlusion](https://mlsanity.com/paper/2006.11366/) (2020)
- [MaskedFace-Net -- A Dataset of Correctly/Incorrectly Masked Face Images in the Context of COVID-19](https://mlsanity.com/paper/2008.08016/) (2020)
- [High Resolution Face Editing with Masked GAN Latent Code Optimization](https://mlsanity.com/paper/2103.11135/) (2021)
- [Robust LSTM-Autoencoders for Face De-Occlusion in the Wild](https://mlsanity.com/paper/1612.08534/) (2016)
- [Unsupervised Eyeglasses Removal in the Wild](https://mlsanity.com/paper/1909.06989/) (2019)
- [Enhancing Convolutional Neural Networks for Face Recognition with Occlusion Maps and Batch Triplet Loss](https://mlsanity.com/paper/1707.07923/) (2017)
- [Contrastive Attention Network with Dense Field Estimation for Face Completion](https://mlsanity.com/paper/2112.10310/) (2021)
- [Learning Disentangling and Fusing Networks for Face Completion Under Structured Occlusions](https://mlsanity.com/paper/1712.04646/) (2017)
- [Robust and Low-Rank Representation for Fast Face Identification with Occlusions](https://mlsanity.com/paper/1605.02266/) (2016)
- [Semantic and Geometric Unfolding of StyleGAN Latent Space](https://mlsanity.com/paper/2107.04481/) (2021)

## Abstract (arXiv)

The pandemic of these very recent years has led to a dramatic increase in people wearing protective masks in public venues. This poses obvious challenges to the pervasive use of face recognition technology that now is suffering a decline in performance. One way to address the problem is to revert to face recovery methods as a preprocessing step. Current approaches to face reconstruction and manipulation leverage the ability to model the face manifold, but tend to be generic. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask. We do so by designing a specialized GAN inversion method, based on an appropriate set of losses for learning an unmasking encoder. With extensive experiments, we show that the approach is effective at unmasking face images. In addition, we also show that the identity information is preserved sufficiently well to improve face verification performance based on several face recognition benchmark datasets.