# LVFace: Progressive Cluster Optimization for Large Vision Models in Face Recognition

arXiv: https://arxiv.org/abs/2501.13420 · HTML: https://mlsanity.com/paper/2501.13420/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 50 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Builds on (most cited first, 30 of 35)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set](https://mlsanity.com/paper/1903.08527/) (2019)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [Additive Margin Softmax for Face Verification](https://mlsanity.com/paper/1801.05599/) (2018)
- [Adaptive Input Representations for Neural Language Modeling](https://mlsanity.com/paper/1809.10853/) (2018)
- [End-to-End Dense Video Captioning with Masked Transformer](https://mlsanity.com/paper/1804.00819/) (2018)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [AdaCos: Adaptively Scaling Cosine Logits for Effectively Learning Deep Face Representations](https://mlsanity.com/paper/1905.00292/) (2019)
- [Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC](https://mlsanity.com/paper/2203.15565/) (2022)
- [Deep Tree Learning for Zero-shot Face Anti-Spoofing](https://mlsanity.com/paper/1904.02860/) (2019)
- [Face Animation with an Attribute-Guided Diffusion Model](https://mlsanity.com/paper/2304.03199/) (2023)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)
- [WebFace260M: A Benchmark for Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2204.10149/) (2022)
- [Masked Face Recognition Challenge: The InsightFace Track Report](https://mlsanity.com/paper/2108.08191/) (2021)
- [CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofing](https://mlsanity.com/paper/2403.14333/) (2024)
- [FitMe: Deep Photorealistic 3D Morphable Model Avatars](https://mlsanity.com/paper/2305.09641/) (2023)
- [FNeVR: Neural Volume Rendering for Face Animation](https://mlsanity.com/paper/2209.10340/) (2022)
- [TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Security](https://mlsanity.com/paper/2308.10133/) (2023)
- [Part-based Face Recognition with Vision Transformers](https://mlsanity.com/paper/2212.00057/) (2022)

## Built on by (most cited first, 12 of 12)

- [ViT-FIQA: Assessing Face Image Quality using Vision Transformers](https://mlsanity.com/paper/2508.13957/) (2025)
- [OmniRefiner: Reinforcement-Guided Local Diffusion Refinement](https://mlsanity.com/paper/2511.19990/) (2025)
- [LocalScore: Local Density-Aware Similarity Scoring for Biometrics](https://mlsanity.com/paper/2602.01012/) (2026)
- [SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception](https://mlsanity.com/paper/2608.10497/) (2026)
- [FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Time](https://mlsanity.com/paper/2510.23515/) (2025)
- [Employing Vision-Language Models for Face Image Quality Assessment](https://mlsanity.com/paper/2605.17489/) (2026)
- [DriveFace: A Cross-Spectral Through-Glass Face Dataset for On-the-Move Vehicular Border Control](https://mlsanity.com/paper/2607.13515/) (2026)
- [GaitFace: A Multimodal Dataset for Long-Range Person Identification](https://mlsanity.com/paper/2607.23542/) (2026)
- [Breaking High Confidence: Practical Face Impersonation under High-Security Thresholds](https://mlsanity.com/paper/2608.20884/) (2026)
- [Vision Language Model Fusion for Explainable Face Recognition](https://mlsanity.com/paper/2608.24430/) (2026)
- [Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras](https://mlsanity.com/paper/2609.06928/) (2026)
- [Towards Robust Classroom Attendance: A Comprehensive Evaluation of Face Detection and Recognition Models](https://mlsanity.com/paper/2609.22750/) (2026)

## Abstract (arXiv)

Vision Transformers (ViTs) have revolutionized large-scale visual modeling, yet remain underexplored in face recognition (FR) where CNNs still dominate. We identify a critical bottleneck: CNN-inspired training paradigms fail to unlock ViT's potential, leading to suboptimal performance and convergence instability.To address this challenge, we propose LVFace, a ViT-based FR model that integrates Progressive Cluster Optimization (PCO) to achieve superior results. Specifically, PCO sequentially applies negative class sub-sampling (NCS) for robust and fast feature alignment from random initialization, feature expectation penalties for centroid stabilization, performing cluster boundary refinement through full-batch training without NCS constraints. LVFace establishes a new state-of-the-art face recognition baseline, surpassing leading approaches such as UniFace and TopoFR across multiple benchmarks. Extensive experiments demonstrate that LVFace delivers consistent performance gains, while exhibiting scalability to large-scale datasets and compatibility with mainstream VLMs and LLMs. Notably, LVFace secured 1st place in the ICCV 2021 Masked Face Recognition (MFR)-Ongoing Challenge (March 2025), proving its efficacy in real-world scenarios. Project is available at https://github.com/bytedance/LVFace.