# Face Clustering: Representation and Pairwise Constraints

arXiv: https://arxiv.org/abs/1706.05067 · HTML: https://mlsanity.com/paper/1706.05067/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2017.

## Built on by (most cited first, 14 of 14)

- [Structure-Aware Face Clustering on a Large-Scale Graph with bf107 Nodes](https://mlsanity.com/paper/2103.13225/) (2021)
- [Ada-NETS: Face Clustering via Adaptive Neighbour Discovery in the Structure Space](https://mlsanity.com/paper/2202.03800/) (2022)
- [Learn to Cluster Faces via Pairwise Classification](https://mlsanity.com/paper/2205.13117/) (2022)
- [Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Framework](https://mlsanity.com/paper/2406.10366/) (2024)
- [FaceMap: Towards Unsupervised Face Clustering via Map Equation](https://mlsanity.com/paper/2203.10090/) (2022)
- [Self-supervised Video-centralised Transformer for Video Face Clustering](https://mlsanity.com/paper/2203.13166/) (2022)
- [Robust Graph Structure Learning via Multiple Statistical Tests](https://mlsanity.com/paper/2210.03956/) (2022)
- [Efficient Large-Scale Face Clustering Using an Online Mixture of Gaussians](https://mlsanity.com/paper/2103.17272/) (2021)
- [Image Clustering using Restricted Boltzman Machine](https://mlsanity.com/paper/2312.13845/) (2023)
- [An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?](https://mlsanity.com/paper/2211.05183/) (2022)
- [Deep Multi-View Semi-Supervised Clustering with Sample Pairwise Constraints](https://mlsanity.com/paper/2206.04949/) (2022)
- [Face Clustering for Connection Discovery from Event Images](https://mlsanity.com/paper/2309.01092/) (2023)
- [Enhancing Noise Resilience in Face Clustering via Sparse Differential Transformer](https://mlsanity.com/paper/2512.22612/) (2025)
- [Fine-grained Fragment Retrieval in Multi-modal Long-form Dialogues](https://mlsanity.com/paper/2606.04591/) (2026)

## Abstract (arXiv)

Clustering face images according to their identity has two important applications: (i) grouping a collection of face images when no external labels are associated with images, and (ii) indexing for efficient large scale face retrieval. The clustering problem is composed of two key parts: face representation and choice of similarity for grouping faces. We first propose a representation based on ResNet, which has been shown to perform very well in image classification problems. Given this representation, we design a clustering algorithm, Conditional Pairwise Clustering (ConPaC), which directly estimates the adjacency matrix only based on the similarity between face images. This allows a dynamic selection of number of clusters and retains pairwise similarity between faces. ConPaC formulates the clustering problem as a Conditional Random Field (CRF) model and uses Loopy Belief Propagation to find an approximate solution for maximizing the posterior probability of the adjacency matrix. Experimental results on two benchmark face datasets (LFW and IJB-B) show that ConPaC outperforms well known clustering algorithms such as k-means, spectral clustering and approximate rank-order. Additionally, our algorithm can naturally incorporate pairwise constraints to obtain a semi-supervised version that leads to improved clustering performance. We also propose an k-NN variant of ConPaC, which has a linear time complexity given a k-NN graph, suitable for large datasets.