# VariFace: Fair and Diverse Synthetic Dataset Generation for Face Recognition arXiv: https://arxiv.org/abs/2412.06235 ยท HTML: https://mlsanity.com/paper/2412.06235/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 60 of its 82 references that are in the map. - First posted on arXiv 9 Dec 2024. ## Builds on (most cited first, 30 of 60) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019) - [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020) - [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022) - [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017) - [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021) - [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016) - [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014) - [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023) - [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020) - [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020) - [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022) - [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021) - [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020) - [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022) - [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018) - [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022) - [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018) - [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023) - [Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/1708.08197/) (2017) - [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021) - [Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning](https://mlsanity.com/paper/2004.11660/) (2020) - [The MegaFace Benchmark: 1 Million Faces for Recognition at Scale](https://mlsanity.com/paper/1512.00596/) (2015) ## Built on by (most cited first, 8 of 8) - [Beyond Real Faces: Synthetic Datasets Can Achieve Reliable Recognition Performance without Privacy Compromise](https://mlsanity.com/paper/2510.17372/) (2025) - [Investigation of Accuracy and Bias in Face Recognition Trained with Synthetic Data](https://mlsanity.com/paper/2507.20782/) (2025) - [ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition](https://mlsanity.com/paper/2506.10226/) (2025) - [A Comparative Study on Synthetic Facial Data Generation Techniques for Face Recognition](https://mlsanity.com/paper/2512.05928/) (2025) - [Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding](https://mlsanity.com/paper/2512.08981/) (2025) - [On Applicability of Synthetic Datasets for Facial Expression Recognition](https://mlsanity.com/paper/2605.17483/) (2026) - [Benchmarking Face Recognition without Real Faces](https://mlsanity.com/paper/2607.14932/) (2026) - [On the Use of Synthetic Data for Threshold Calibration in Face Recognition: Performance and Security Implications for Border Control Systems](https://mlsanity.com/paper/2607.25990/) (2026) ## Abstract (arXiv) The use of large-scale, web-scraped datasets to train face recognition models has raised significant privacy and bias concerns. Synthetic methods mitigate these concerns and provide scalable and controllable face generation to enable fair and accurate face recognition. However, existing synthetic datasets display limited intraclass and interclass diversity and do not match the face recognition performance obtained using real datasets. Here, we propose VariFace, a two-stage diffusion-based pipeline to create fair and diverse synthetic face datasets to train face recognition models. Specifically, we introduce three methods: Face Recognition Consistency to refine demographic labels, Face Vendi Score Guidance to improve interclass diversity, and Divergence Score Conditioning to balance the identity preservation-intraclass diversity trade-off. When constrained to the same dataset size, VariFace considerably outperforms previous synthetic datasets (0.9200 rightarrow 0.9405) and achieves comparable performance to face recognition models trained with real data (Real Gap = -0.0065). In an unconstrained setting, VariFace not only consistently achieves better performance compared to previous synthetic methods across dataset sizes but also, for the first time, outperforms the real dataset (CASIA-WebFace) across six evaluation datasets. This sets a new state-of-the-art performance with an average face verification accuracy of 0.9567 (Real Gap = +0.0097) across LFW, CFP-FP, CPLFW, AgeDB, and CALFW datasets and 0.9366 (Real Gap = +0.0380) on the RFW dataset.