# VideoClusterNet: Self-Supervised and Adaptive Face Clustering For Videos

arXiv: https://arxiv.org/abs/2407.12214 · HTML: https://mlsanity.com/paper/2407.12214/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 65 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Builds on (most cited first, 25 of 25)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [MovieQA: Understanding Stories in Movies through Question-Answering](https://mlsanity.com/paper/1512.02902/) (2015)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [General Facial Representation Learning in a Visual-Linguistic Manner](https://mlsanity.com/paper/2112.03109/) (2021)
- [Movie Description](https://mlsanity.com/paper/1605.03705/) (2016)
- [SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness](https://mlsanity.com/paper/2003.09373/) (2020)
- [A Comprehensive Survey on Deep Clustering: Taxonomy, Challenges, and Future Directions](https://mlsanity.com/paper/2206.07579/) (2022)
- [MovieGraphs: Towards Understanding Human-Centric Situations from Videos](https://mlsanity.com/paper/1712.06761/) (2017)
- [Mugs: A Multi-Granular Self-Supervised Learning Framework](https://mlsanity.com/paper/2203.14415/) (2022)
- [Clustering based Contrastive Learning for Improving Face Representations](https://mlsanity.com/paper/2004.02195/) (2020)
- [Video Face Clustering with Unknown Number of Clusters](https://mlsanity.com/paper/1908.03381/) (2019)
- [Video Face Recognition: Component-wise Feature Aggregation Network (C-FAN)](https://mlsanity.com/paper/1902.07327/) (2019)
- [Self-Supervised Learning of Face Representations for Video Face Clustering](https://mlsanity.com/paper/1903.01000/) (2019)
- [Unsupervised Learning of Face Representations](https://mlsanity.com/paper/1803.01260/) (2018)
- [Robust Character Labeling in Movie Videos: Data Resources and Self-supervised Feature Adaptation](https://mlsanity.com/paper/2008.11289/) (2020)
- [Self-supervised Video-centralised Transformer for Video Face Clustering](https://mlsanity.com/paper/2203.13166/) (2022)
- [Unique Faces Recognition in Videos](https://mlsanity.com/paper/2006.05713/) (2020)

## Built on by (most cited first, 1 of 1)

- [Feature-Centric Unsupervised Node Representation Learning Without Homophily Assumption](https://mlsanity.com/paper/2512.15112/) (2025)

## Abstract (arXiv)

With the rise of digital media content production, the need for analyzing movies and TV series episodes to locate the main cast of characters precisely is gaining importance.Specifically, Video Face Clustering aims to group together detected video face tracks with common facial identities. This problem is very challenging due to the large range of pose, expression, appearance, and lighting variations of a given face across video frames. Generic pre-trained Face Identification (ID) models fail to adapt well to the video production domain, given its high dynamic range content and also unique cinematic style. Furthermore, traditional clustering algorithms depend on hyperparameters requiring individual tuning across datasets. In this paper, we present a novel video face clustering approach that learns to adapt a generic face ID model to new video face tracks in a fully self-supervised fashion. We also propose a parameter-free clustering algorithm that is capable of automatically adapting to the finetuned model's embedding space for any input video. Due to the lack of comprehensive movie face clustering benchmarks, we also present a first-of-kind movie dataset: MovieFaceCluster. Our dataset is handpicked by film industry professionals and contains extremely challenging face ID scenarios. Experiments show our method's effectiveness in handling difficult mainstream movie scenes on our benchmark dataset and state-of-the-art performance on traditional TV series datasets.