# SegFace: Face Segmentation of Long-Tail Classes

arXiv: https://arxiv.org/abs/2412.08647 · HTML: https://mlsanity.com/paper/2412.08647/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 36 references that are in the map.
- First posted on arXiv 11 Dec 2024.

## Builds on (most cited first, 6 of 6)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Face Parsing via Recurrent Propagation](https://mlsanity.com/paper/1708.01936/) (2017)

## Built on by (most cited first, 18 of 18)

- [FaceXBench: Evaluating Multimodal LLMs on Face Understanding](https://mlsanity.com/paper/2501.10360/) (2025)
- [BiggerGait: Unlocking Gait Recognition with Layer-wise Representations from Large Vision Models](https://mlsanity.com/paper/2505.18132/) (2025)
- [Symmetrical Flow Matching: Unified Image Generation, Segmentation, and Classification with Score-Based Generative Models](https://mlsanity.com/paper/2506.10634/) (2025)
- [Dynamic Dictionary Learning for Remote Sensing Image Segmentation](https://mlsanity.com/paper/2503.06683/) (2025)
- [Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation](https://mlsanity.com/paper/2509.00428/) (2025)
- [DisFaceRep: Representation Disentanglement for Co-occurring Facial Components in Weakly Supervised Face Parsing](https://mlsanity.com/paper/2508.01250/) (2025)
- [PVLM: Parsing-Aware Vision Language Model with Dynamic Contrastive Learning for Zero-Shot Deepfake Attribution](https://mlsanity.com/paper/2504.14129/) (2025)
- [ExpertGen: Training-Free Expert Guidance for Controllable Text-to-Face Generation](https://mlsanity.com/paper/2505.17256/) (2025)
- [PixelArena: A benchmark for Pixel-Precision Visual Intelligence](https://mlsanity.com/paper/2512.16303/) (2025)
- [Surveillance Facial Image Quality Assessment: A Multi-dimensional Dataset and Lightweight Model](https://mlsanity.com/paper/2602.07403/) (2026)
- [Reframing Long-Tailed Learning via Loss Landscape Geometry](https://mlsanity.com/paper/2603.21217/) (2026)
- [S3POT: Contrast-Driven Face Occlusion Segmentation via Self-Supervised Prompt Learning](https://mlsanity.com/paper/2602.00635/) (2026)
- [IdentiFace: Multi-Modal Iterative Diffusion Framework for Identifiable Suspect Face Generation in Crime Investigations](https://mlsanity.com/paper/2605.00526/) (2026)
- [FaceMoE: Mixture of Experts for Low-Resolution Face Recognition](https://mlsanity.com/paper/2606.32040/) (2026)
- [DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection](https://mlsanity.com/paper/2609.35096/) (2026)
- [Domain-adaptive Zero-Shot Image Enhancement via Locality-Constrained Diffusion Guidance](https://mlsanity.com/paper/2609.35289/) (2026)
- [OFBD: Object-Focused Background Debiasing for Long-Tailed Learning](https://mlsanity.com/paper/2609.37331/) (2026)
- [DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding](https://mlsanity.com/paper/2610.17015/) (2026)

## Abstract (arXiv)

Face parsing refers to the semantic segmentation of human faces into key facial regions such as eyes, nose, hair, etc. It serves as a prerequisite for various advanced applications, including face editing, face swapping, and facial makeup, which often require segmentation masks for classes like eyeglasses, hats, earrings, and necklaces. These infrequently occurring classes are called long-tail classes, which are overshadowed by more frequently occurring classes known as head classes. Existing methods, primarily CNN-based, tend to be dominated by head classes during training, resulting in suboptimal representation for long-tail classes. Previous works have largely overlooked the problem of poor segmentation performance of long-tail classes. To address this issue, we propose SegFace, a simple and efficient approach that uses a lightweight transformer-based model which utilizes learnable class-specific tokens. The transformer decoder leverages class-specific tokens, allowing each token to focus on its corresponding class, thereby enabling independent modeling of each class. The proposed approach improves the performance of long-tail classes, thereby boosting overall performance. To the best of our knowledge, SegFace is the first work to employ transformer models for face parsing. Moreover, our approach can be adapted for low-compute edge devices, achieving 95.96 FPS. We conduct extensive experiments demonstrating that SegFace significantly outperforms previous state-of-the-art models, achieving a mean F1 score of 88.96 (+2.82) on the CelebAMask-HQ dataset and 93.03 (+0.65) on the LaPa dataset. Code: https://github.com/Kartik-3004/SegFace