# Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN

arXiv: https://arxiv.org/abs/2212.09098 · HTML: https://mlsanity.com/paper/2212.09098/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 18 Dec 2022.

## Builds on (most cited first, 30 of 30)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation](https://mlsanity.com/paper/1606.02147/) (2016)
- [ICNet for Real-Time Semantic Segmentation on High-Resolution Images](https://mlsanity.com/paper/1704.08545/) (2017)
- [Fine-Grained Head Pose Estimation Without Keypoints](https://mlsanity.com/paper/1710.00925/) (2017)
- [Look at Boundary: A Boundary-Aware Face Alignment Algorithm](https://mlsanity.com/paper/1805.10483/) (2018)
- [Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression](https://mlsanity.com/paper/1904.07399/) (2019)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition](https://mlsanity.com/paper/1712.04695/) (2017)
- [Style Aggregated Network for Facial Landmark Detection](https://mlsanity.com/paper/1803.04108/) (2018)
- [Not All Pixels Are Equal: Difficulty-aware Semantic Segmentation via Deep Layer Cascade](https://mlsanity.com/paper/1704.01344/) (2017)
- [Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors](https://mlsanity.com/paper/1807.00966/) (2018)
- [Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning](https://mlsanity.com/paper/2203.06541/) (2022)
- [Face Parsing with RoI Tanh-Warping](https://mlsanity.com/paper/1906.01342/) (2019)
- [RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer](https://mlsanity.com/paper/2210.07124/) (2022)
- [PFLD: A Practical Facial Landmark Detector](https://mlsanity.com/paper/1902.10859/) (2019)
- [Robust LSTM-Autoencoders for Face De-Occlusion in the Wild](https://mlsanity.com/paper/1612.08534/) (2016)
- [Interlinked Convolutional Neural Networks for Face Parsing](https://mlsanity.com/paper/1806.02479/) (2018)
- [AGRNet: Adaptive Graph Representation Learning and Reasoning for Face Parsing](https://mlsanity.com/paper/2101.07034/) (2021)
- [Face Parsing via Recurrent Propagation](https://mlsanity.com/paper/1708.01936/) (2017)
- [Real-Time Facial Segmentation and Performance Capture from RGB Input](https://mlsanity.com/paper/1604.02647/) (2016)
- [A CNN Cascade for Landmark Guided Semantic Part Segmentation](https://mlsanity.com/paper/1609.09642/) (2016)
- [Deep Learning Based 3D Point Cloud Regression for Estimating Forest Biomass](https://mlsanity.com/paper/2112.11335/) (2021)
- [LOTR: Face Landmark Localization Using Localization Transformer](https://mlsanity.com/paper/2109.10057/) (2021)

## Built on by (most cited first, 16 of 16)

- [ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preserving](https://mlsanity.com/paper/2404.16771/) (2024)
- [Human Motion Instruction Tuning](https://mlsanity.com/paper/2411.16805/) (2024)
- [CPSeg: Finer-grained Image Semantic Segmentation via Chain-of-Thought Language Prompting](https://mlsanity.com/paper/2310.16069/) (2023)
- [Credit Card Fraud Detection Using Advanced Transformer Model](https://mlsanity.com/paper/2406.03733/) (2024)
- [Rumor Detection with a novel graph neural network approach](https://mlsanity.com/paper/2403.16206/) (2024)
- [CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applications](https://mlsanity.com/paper/2408.03703/) (2024)
- [RAM: Recover Any 3D Human Motion in-the-Wild](https://mlsanity.com/paper/2603.19929/) (2026)
- [Segment Any Building](https://mlsanity.com/paper/2310.01164/) (2023)
- [Wav2Sem: Plug-and-Play Audio Semantic Decoupling for 3D Speech-Driven Facial Animation](https://mlsanity.com/paper/2505.23290/) (2025)
- [You Only Speak Once to See](https://mlsanity.com/paper/2409.18372/) (2024)
- [Edge Aware Learning for 3D Point Cloud](https://mlsanity.com/paper/2309.13472/) (2023)
- [End-To-End Training and Testing Gamification Framework to Learn Human Highway Driving](https://mlsanity.com/paper/2404.10849/) (2024)
- [Tree Counting by Bridging 3D Point Clouds with Imagery](https://mlsanity.com/paper/2403.01932/) (2024)
- [CNSocialDepress: A Chinese Social Media Dataset for Depression Risk Detection and Structured Analysis](https://mlsanity.com/paper/2510.11233/) (2025)
- [Occlusion-Aware Deep Convolutional Neural Network via Homogeneous Tanh-transforms for Face Parsing](https://mlsanity.com/paper/2308.15323/) (2023)
- [MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder](https://mlsanity.com/paper/2403.04626/) (2024)

## Abstract (arXiv)

Fine-grained semantic segmentation of a person's face and head, including facial parts and head components, has progressed a great deal in recent years. However, it remains a challenging task, whereby considering ambiguous occlusions and large pose variations are particularly difficult. To overcome these difficulties, we propose a novel framework termed Mask-FPAN. It uses a de-occlusion module that learns to parse occluded faces in a semi-supervised way. In particular, face landmark localization, face occlusionstimations, and detected head poses are taken into account. A 3D morphable face model combined with the UV GAN improves the robustness of 2D face parsing. In addition, we introduce two new datasets named FaceOccMask-HQ and CelebAMaskOcc-HQ for face paring work. The proposed Mask-FPAN framework addresses the face parsing problem in the wild and shows significant performance improvements with MIOU from 0.7353 to 0.9013 compared to the state-of-the-art on challenging face datasets.