# Cluster and Aggregate: Face Recognition with Large Probe Set

arXiv: https://arxiv.org/abs/2210.10864 · HTML: https://mlsanity.com/paper/2210.10864/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 62 references that are in the map.
- First posted on arXiv 19 Oct 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Conditional Positional Encodings for Vision Transformers](https://mlsanity.com/paper/2102.10882/) (2021)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [VidTr: Video Transformer Without Convolutions](https://mlsanity.com/paper/2104.11746/) (2021)
- [L2-constrained Softmax Loss for Discriminative Face Verification](https://mlsanity.com/paper/1703.09507/) (2017)
- [Memory Transformer](https://mlsanity.com/paper/2006.11527/) (2020)
- [SRM : A Style-based Recalibration Module for Convolutional Neural Networks](https://mlsanity.com/paper/1903.10829/) (2019)
- [Neural Aggregation Network for Video Face Recognition](https://mlsanity.com/paper/1603.05474/) (2016)
- [Quality Aware Network for Set to Set Recognition](https://mlsanity.com/paper/1704.03373/) (2017)

## Built on by (most cited first, 22 of 22)

- [DCFace: Synthetic Face Generation with Dual Condition Diffusion Model](https://mlsanity.com/paper/2304.07060/) (2023)
- [KeyPoint Relative Position Encoding for Face Recognition](https://mlsanity.com/paper/2403.14852/) (2024)
- [FarSight: A Physics-Driven Whole-Body Biometric System at Large Distance and Altitude](https://mlsanity.com/paper/2306.17206/) (2023)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [SapiensID: Foundation for Human Recognition](https://mlsanity.com/paper/2504.04708/) (2025)
- [Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait](https://mlsanity.com/paper/2505.04616/) (2025)
- [Open-Set Biometrics: Beyond Good Closed-Set Models](https://mlsanity.com/paper/2407.16133/) (2024)
- [A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition](https://mlsanity.com/paper/2508.00053/) (2025)
- [CoNAN: Conditional Neural Aggregation Network For Unconstrained Face Feature Fusion](https://mlsanity.com/paper/2307.10237/) (2023)
- [LocalScore: Local Density-Aware Similarity Scoring for Biometrics](https://mlsanity.com/paper/2602.01012/) (2026)
- [FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition](https://mlsanity.com/paper/2603.26908/) (2026)
- [Are you In or Out (of gallery)? Wisdom from the Same-Identity Crowd](https://mlsanity.com/paper/2508.06357/) (2025)
- [FaceCoresetNet: Differentiable Coresets for Face Set Recognition](https://mlsanity.com/paper/2308.14075/) (2023)
- [Probabilistic Quantum SVM Training on Ising Machine](https://mlsanity.com/paper/2503.16363/) (2025)
- [IDSelect: A RL-Based Cost-Aware Selection Agent for Video-based Multi-Modal Person Recognition](https://mlsanity.com/paper/2602.18990/) (2026)
- [DaliID: Distortion-Adaptive Learned Invariance for Identification Models](https://mlsanity.com/paper/2302.05753/) (2023)
- [Embedding Aggregation for Forensic Facial Comparison](https://mlsanity.com/paper/2305.00352/) (2023)
- [Quo Vadis RankList-based System in Face Recognition?](https://mlsanity.com/paper/2410.01498/) (2024)
- [Sparsity-Inducing Divergence Losses for Biometric Verification](https://mlsanity.com/paper/2606.31664/) (2026)
- [FaceMoE: Mixture of Experts for Low-Resolution Face Recognition](https://mlsanity.com/paper/2606.32040/) (2026)
- [Rank-1 Identity Consensus Predicts Gallery Enrollment in 1:N Face Matching More Accurately than Score Thresholding](https://mlsanity.com/paper/2607.12903/) (2026)
- [Video-Based Palm-Vein Authentication under Challenging Conditions](https://mlsanity.com/paper/2609.02776/) (2026)

## Abstract (arXiv)

Feature fusion plays a crucial role in unconstrained face recognition where inputs (probes) comprise of a set of N low quality images whose individual qualities vary. Advances in attention and recurrent modules have led to feature fusion that can model the relationship among the images in the input set. However, attention mechanisms cannot scale to large N due to their quadratic complexity and recurrent modules suffer from input order sensitivity. We propose a two-stage feature fusion paradigm, Cluster and Aggregate, that can both scale to large N and maintain the ability to perform sequential inference with order invariance. Specifically, Cluster stage is a linear assignment of N inputs to M global cluster centers, and Aggregation stage is a fusion over M clustered features. The clustered features play an integral role when the inputs are sequential as they can serve as a summarization of past features. By leveraging the order-invariance of incremental averaging operation, we design an update rule that achieves batch-order invariance, which guarantees that the contributions of early image in the sequence do not diminish as time steps increase. Experiments on IJB-B and IJB-S benchmark datasets show the superiority of the proposed two-stage paradigm in unconstrained face recognition. Code and pretrained models are available in https://github.com/mk-minchul/caface