# FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition

arXiv: https://arxiv.org/abs/2112.12496 · HTML: https://mlsanity.com/paper/2112.12496/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 41 references that are in the map.
- First posted on arXiv 23 Dec 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection](https://mlsanity.com/paper/1907.09693/) (2019)
- [Think Locally, Act Globally: Federated Learning with Local and Global Representations](https://mlsanity.com/paper/2001.01523/) (2020)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Personalized Federated Learning: A Meta-Learning Approach](https://mlsanity.com/paper/2002.07948/) (2020)
- [Tighter Theory for Local SGD on Identical and Heterogeneous Data](https://mlsanity.com/paper/1909.04746/) (2019)
- [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021)
- [Federated Meta-Learning with Fast Convergence and Efficient Communication](https://mlsanity.com/paper/1802.07876/) (2018)
- [On the Convergence of Local Descent Methods in Federated Learning](https://mlsanity.com/paper/1910.14425/) (2019)
- [Survey of Personalization Techniques for Federated Learning](https://mlsanity.com/paper/2003.08673/) (2020)
- [Salvaging Federated Learning by Local Adaptation](https://mlsanity.com/paper/2002.04758/) (2020)
- [Federated Learning with Only Positive Labels](https://mlsanity.com/paper/2004.10342/) (2020)
- [Performance Optimization for Federated Person Re-identification via Benchmark Analysis](https://mlsanity.com/paper/2008.11560/) (2020)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)

## Built on by (most cited first, 16 of 16)

- [Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation](https://mlsanity.com/paper/2308.15367/) (2023)
- [ProtoFL: Unsupervised Federated Learning via Prototypical Distillation](https://mlsanity.com/paper/2307.12450/) (2023)
- [COALA: A Practical and Vision-Centric Federated Learning Platform](https://mlsanity.com/paper/2407.16560/) (2024)
- [Rethinking Personalized Federated Learning with Clustering-based Dynamic Graph Propagation](https://mlsanity.com/paper/2401.15874/) (2024)
- [A Survey of Face Recognition](https://mlsanity.com/paper/2212.13038/) (2022)
- [Towards Zero-trust Security for the Metaverse](https://mlsanity.com/paper/2302.08885/) (2023)
- [Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations](https://mlsanity.com/paper/2306.03600/) (2023)
- [Trustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune](https://mlsanity.com/paper/2402.16911/) (2024)
- [PAFedFV: Personalized and Asynchronous Federated Learning for Finger Vein Recognition](https://mlsanity.com/paper/2404.13237/) (2024)
- [Uncertainty-Aware Label Refinement on Hypergraphs for Personalized Federated Facial Expression Recognition](https://mlsanity.com/paper/2501.01816/) (2025)
- [Federated Learning Method for Preserving Privacy in Face Recognition System](https://mlsanity.com/paper/2403.05344/) (2024)
- [Adaptive Hybrid Masking Strategy for Privacy-Preserving Face Recognition Against Model Inversion Attack](https://mlsanity.com/paper/2403.10558/) (2024)
- [AdaFedFR: Federated Face Recognition with Adaptive Inter-Class Representation Learning](https://mlsanity.com/paper/2405.13467/) (2024)
- [Federated Learning for Face Recognition via Intra-subject Self-supervised Learning](https://mlsanity.com/paper/2407.16289/) (2024)
- [Meta-Learning for Federated Face Recognition in Imbalanced Data Regimes](https://mlsanity.com/paper/2408.16003/) (2024)
- [Prompt Estimation from Prototypes for Federated Prompt Tuning of Vision Transformers](https://mlsanity.com/paper/2510.25372/) (2025)

## Abstract (arXiv)

Current state-of-the-art deep learning based face recognition (FR) models require a large number of face identities for central training. However, due to the growing privacy awareness, it is prohibited to access the face images on user devices to continually improve face recognition models. Federated Learning (FL) is a technique to address the privacy issue, which can collaboratively optimize the model without sharing the data between clients. In this work, we propose a FL based framework called FedFR to improve the generic face representation in a privacy-aware manner. Besides, the framework jointly optimizes personalized models for the corresponding clients via the proposed Decoupled Feature Customization module. The client-specific personalized model can serve the need of optimized face recognition experience for registered identities at the local device. To the best of our knowledge, we are the first to explore the personalized face recognition in FL setup. The proposed framework is validated to be superior to previous approaches on several generic and personalized face recognition benchmarks with diverse FL scenarios. The source codes and our proposed personalized FR benchmark under FL setup are available at https://github.com/jackie840129/FedFR.