# Learning to Cluster Faces via Transformer

arXiv: https://arxiv.org/abs/2104.11502 · HTML: https://mlsanity.com/paper/2104.11502/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 23 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering](https://mlsanity.com/paper/1610.04794/) (2016)
- [Linkage Based Face Clustering via Graph Convolution Network](https://mlsanity.com/paper/1903.11306/) (2019)
- [Learning to Cluster Faces on an Affinity Graph](https://mlsanity.com/paper/1904.02749/) (2019)
- [Learning to Cluster Faces via Confidence and Connectivity Estimation](https://mlsanity.com/paper/2004.00445/) (2020)
- [Clustering Millions of Faces by Identity](https://mlsanity.com/paper/1604.00989/) (2016)
- [Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition](https://mlsanity.com/paper/1809.01407/) (2018)
- [A Proximity-Aware Hierarchical Clustering of Faces](https://mlsanity.com/paper/1703.04835/) (2017)

## Built on by (most cited first, 5 of 5)

- [Relation-Aware Distribution Representation Network for Person Clustering with Multiple Modalities](https://mlsanity.com/paper/2308.00588/) (2023)
- [An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?](https://mlsanity.com/paper/2211.05183/) (2022)
- [Face Clustering via Early Stopping and Edge Recall](https://mlsanity.com/paper/2408.13431/) (2024)
- [Learn to Cluster Faces with Better Subgraphs](https://mlsanity.com/paper/2304.10831/) (2023)
- [Enhancing Noise Resilience in Face Clustering via Sparse Differential Transformer](https://mlsanity.com/paper/2512.22612/) (2025)

## Abstract (arXiv)

Face clustering is a useful tool for applications like automatic face annotation and retrieval. The main challenge is that it is difficult to cluster images from the same identity with different face poses, occlusions, and image quality. Traditional clustering methods usually ignore the relationship between individual images and their neighbors which may contain useful context information. In this paper, we repurpose the well-known Transformer and introduce a Face Transformer for supervised face clustering. In Face Transformer, we decompose the face clustering into two steps: relation encoding and linkage predicting. Specifically, given a face image, a relation encoder module aggregates local context information from its neighbors and a linkage predictor module judges whether a pair of images belong to the same cluster or not. In the local linkage graph view, Face Transformer can generate more robust node and edge representations compared to existing methods. Experiments on both MS-Celeb-1M and DeepFashion show that our method achieves state-of-the-art performance, e.g., 91.12% in pairwise F-score on MS-Celeb-1M.