# An Empirical Study on Clustering Pretrained Embeddings: Is Deep Strictly Better?

arXiv: https://arxiv.org/abs/2211.05183 · HTML: https://mlsanity.com/paper/2211.05183/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 54 references that are in the map.
- First posted on arXiv 9 Nov 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015)
- [A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses](https://mlsanity.com/paper/2003.08983/) (2020)
- [L2-constrained Softmax Loss for Discriminative Face Verification](https://mlsanity.com/paper/1703.09507/) (2017)
- [Linkage Based Face Clustering via Graph Convolution Network](https://mlsanity.com/paper/1903.11306/) (2019)
- [Learning to Cluster Faces on an Affinity Graph](https://mlsanity.com/paper/1904.02749/) (2019)
- [Learning to Cluster Faces via Confidence and Connectivity Estimation](https://mlsanity.com/paper/2004.00445/) (2020)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [Clustering Millions of Faces by Identity](https://mlsanity.com/paper/1604.00989/) (2016)
- [A Survey of Inductive Biases for Factorial Representation-Learning](https://mlsanity.com/paper/1612.05299/) (2016)
- [Structure-Aware Face Clustering on a Large-Scale Graph with bf107 Nodes](https://mlsanity.com/paper/2103.13225/) (2021)
- [Consensus-Driven Propagation in Massive Unlabeled Data for Face Recognition](https://mlsanity.com/paper/1809.01407/) (2018)
- [Face Clustering: Representation and Pairwise Constraints](https://mlsanity.com/paper/1706.05067/) (2017)
- [Crystal Loss and Quality Pooling for Unconstrained Face Verification and Recognition](https://mlsanity.com/paper/1804.01159/) (2018)
- [Merge or Not? Learning to Group Faces via Imitation Learning](https://mlsanity.com/paper/1707.03986/) (2017)
- [Learning to Cluster Faces via Transformer](https://mlsanity.com/paper/2104.11502/) (2021)
- [Small-Variance Nonparametric Clustering on the Hypersphere](https://mlsanity.com/paper/1607.06407/) (2016)

## Built on by (most cited first, 1 of 1)

- [Spherical Cauchy Variational Autoencoders: Heavy Angular Tails and Exact KL Evaluation](https://mlsanity.com/paper/2506.21278/) (2025)

## Abstract (arXiv)

Recent research in clustering face embeddings has found that unsupervised, shallow, heuristic-based methods -- including k-means and hierarchical agglomerative clustering -- underperform supervised, deep, inductive methods. While the reported improvements are indeed impressive, experiments are mostly limited to face datasets, where the clustered embeddings are highly discriminative or well-separated by class (Recall@1 above 90% and often nearing ceiling), and the experimental methodology seemingly favors the deep methods. We conduct a large-scale empirical study of 17 clustering methods across three datasets and obtain several robust findings. Notably, deep methods are surprisingly fragile for embeddings with more uncertainty, where they match or even perform worse than shallow, heuristic-based methods. When embeddings are highly discriminative, deep methods do outperform the baselines, consistent with past results, but the margin between methods is much smaller than previously reported. We believe our benchmarks broaden the scope of supervised clustering methods beyond the face domain and can serve as a foundation on which these methods could be improved. To enable reproducibility, we include all necessary details in the appendices, and plan to release the code.