# Uncertainty-Guided Face Matting for Occlusion-Aware Face Transformation

arXiv: https://arxiv.org/abs/2508.03055 · HTML: https://mlsanity.com/paper/2508.03055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Robust High-Resolution Video Matting with Temporal Guidance](https://mlsanity.com/paper/2108.11515/) (2021)
- [MODNet: Real-Time Trimap-Free Portrait Matting via Objective Decomposition](https://mlsanity.com/paper/2011.11961/) (2020)
- [Gaussian YOLOv3: An Accurate and Fast Object Detector Using Localization Uncertainty for Autonomous Driving](https://mlsanity.com/paper/1904.04620/) (2019)
- [Bridging Composite and Real: Towards End-to-end Deep Image Matting](https://mlsanity.com/paper/2010.16188/) (2020)
- [Mask Guided Matting via Progressive Refinement Network](https://mlsanity.com/paper/2012.06722/) (2020)
- [Natural Image Matting via Guided Contextual Attention](https://mlsanity.com/paper/2001.04069/) (2020)
- [On Face Segmentation, Face Swapping, and Face Perception](https://mlsanity.com/paper/1704.06729/) (2017)
- [Deep Automatic Natural Image Matting](https://mlsanity.com/paper/2107.07235/) (2021)
- [Semantic Image Matting](https://mlsanity.com/paper/2104.08201/) (2021)
- [Occlusion Robust Face Recognition Based on Mask Learning with PairwiseDifferential Siamese Network](https://mlsanity.com/paper/1908.06290/) (2019)
- [AlphaGAN: Generative adversarial networks for natural image matting](https://mlsanity.com/paper/1807.10088/) (2018)
- [MatteFormer: Transformer-Based Image Matting via Prior-Tokens](https://mlsanity.com/paper/2203.15662/) (2022)
- [Deep Video Matting via Spatio-Temporal Alignment and Aggregation](https://mlsanity.com/paper/2104.11208/) (2021)
- [Hand Image Understanding via Deep Multi-Task Learning](https://mlsanity.com/paper/2107.11646/) (2021)
- [Attention-guided Temporally Coherent Video Object Matting](https://mlsanity.com/paper/2105.11427/) (2021)
- [PP-Matting: High-Accuracy Natural Image Matting](https://mlsanity.com/paper/2204.09433/) (2022)
- [TransMatting: Enhancing Transparent Objects Matting with Transformers](https://mlsanity.com/paper/2208.03007/) (2022)
- [Delving into High-Quality Synthetic Face Occlusion Segmentation Datasets](https://mlsanity.com/paper/2205.06218/) (2022)
- [Segmentation-Reconstruction-Guided Facial Image De-occlusion](https://mlsanity.com/paper/2112.08022/) (2021)
- [FaceOcc: A Diverse, High-quality Face Occlusion Dataset for Human Face Extraction](https://mlsanity.com/paper/2201.08425/) (2022)

## Abstract (arXiv)

Face filters have become a key element of short-form video content, enabling a wide array of visual effects such as stylization and face swapping. However, their performance often degrades in the presence of occlusions, where objects like hands, hair, or accessories obscure the face. To address this limitation, we introduce the novel task of face matting, which estimates fine-grained alpha mattes to separate occluding elements from facial regions. We further present FaceMat, a trimap-free, uncertainty-aware framework that predicts high-quality alpha mattes under complex occlusions. Our approach leverages a two-stage training pipeline: a teacher model is trained to jointly estimate alpha mattes and per-pixel uncertainty using a negative log-likelihood (NLL) loss, and this uncertainty is then used to guide the student model through spatially adaptive knowledge distillation. This formulation enables the student to focus on ambiguous or occluded regions, improving generalization and preserving semantic consistency. Unlike previous approaches that rely on trimaps or segmentation masks, our framework requires no auxiliary inputs making it well-suited for real-time applications. In addition, we reformulate the matting objective by explicitly treating skin as foreground and occlusions as background, enabling clearer compositing strategies. To support this task, we newly constructed CelebAMat, a large-scale synthetic dataset specifically designed for occlusion-aware face matting. Extensive experiments show that FaceMat outperforms state-of-the-art methods across multiple benchmarks, enhancing the visual quality and robustness of face filters in real-world, unconstrained video scenarios. The source code and CelebAMat dataset are available at https://github.com/hyebin-c/FaceMat.git