# Rethinking Common Assumptions to Mitigate Racial Bias in Face Recognition Datasets arXiv: https://arxiv.org/abs/2109.03229 ยท HTML: https://mlsanity.com/paper/2109.03229/ ## Key facts - Cited by 22 papers in the map (data through 14 Oct 2026). - Builds on 23 of its 43 references that are in the map. - First posted on arXiv 7 Sep 2021. ## Builds on (most cited first, 23 of 23) - [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017) - [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015) - [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019) - [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018) - [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017) - [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017) - [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016) - [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014) - [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018) - [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019) - [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017) - [WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition](https://mlsanity.com/paper/2103.04098/) (2021) - [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019) - [L2-constrained Softmax Loss for Discriminative Face Verification](https://mlsanity.com/paper/1703.09507/) (2017) - [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019) - [Mitigating Face Recognition Bias via Group Adaptive Classifier](https://mlsanity.com/paper/2006.07576/) (2020) - [CASIA-SURF CeFA: A Benchmark for Multi-modal Cross-ethnicity Face Anti-spoofing](https://mlsanity.com/paper/2003.05136/) (2020) - [SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning](https://mlsanity.com/paper/2004.11246/) (2020) - [Analysis of Gender Inequality In Face Recognition Accuracy](https://mlsanity.com/paper/2002.00065/) (2020) - [Exploring Racial Bias within Face Recognition via per-subject Adversarially-Enabled Data Augmentation](https://mlsanity.com/paper/2004.08945/) (2020) - [Domain Balancing: Face Recognition on Long-Tailed Domains](https://mlsanity.com/paper/2003.13791/) (2020) - [Deep Learning for Face Recognition: Pride or Prejudiced?](https://mlsanity.com/paper/1904.01219/) (2019) - [Reducing Geographic Performance Differential for Face Recognition](https://mlsanity.com/paper/2002.12093/) (2020) ## Built on by (most cited first, 22 of 22) - [Studying Bias in GANs through the Lens of Race](https://mlsanity.com/paper/2209.02836/) (2022) - [FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classification](https://mlsanity.com/paper/2207.10888/) (2022) - [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021) - [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025) - [What Should Be Balanced in a "Balanced" Face Recognition Dataset?](https://mlsanity.com/paper/2304.09818/) (2023) - [Mitigating Gender Bias in Face Recognition Using the von Mises-Fisher Mixture Model](https://mlsanity.com/paper/2210.13664/) (2022) - [A Deep Dive into Dataset Imbalance and Bias in Face Identification](https://mlsanity.com/paper/2203.08235/) (2022) - [MixFairFace: Towards Ultimate Fairness via MixFair Adapter in Face Recognition](https://mlsanity.com/paper/2211.15181/) (2022) - [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025) - [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021) - [Explaining Bias in Deep Face Recognition via Image Characteristics](https://mlsanity.com/paper/2208.11099/) (2022) - [An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications](https://mlsanity.com/paper/2301.13476/) (2023) - [Are Commercial Face Detection Models as Biased as Academic Models?](https://mlsanity.com/paper/2201.10047/) (2022) - [MIME: Minority Inclusion for Majority Group Enhancement of AI Performance](https://mlsanity.com/paper/2209.00746/) (2022) - [The More Secure, The Less Equally Usable: Gender and Ethnicity (Un)fairness of Deep Face Recognition along Security Thresholds](https://mlsanity.com/paper/2209.15550/) (2022) - [Co-segmentation Inspired Attention Module for Video-based Computer Vision Tasks](https://mlsanity.com/paper/2111.07370/) (2021) - [(Un)fair Exposure in Deep Face Rankings at a Distance](https://mlsanity.com/paper/2308.11732/) (2023) - [Mitigating Bias in Facial Recognition Systems: Centroid Fairness Loss Optimization](https://mlsanity.com/paper/2504.19370/) (2025) - [Balancing Beyond Discrete Categories: Continuous Demographic Labels for Fair Face Recognition](https://mlsanity.com/paper/2506.01532/) (2025) - [Reliable and Reproducible Demographic Inference for Fairness in Face Analysis](https://mlsanity.com/paper/2510.20482/) (2025) - [Skin Tone in the Wild: A Dataset and Benchmark for 10-tone MST Classification](https://mlsanity.com/paper/2603.02475/) (2026) - [DenseFace: Bias Mitigation in Face Recognition via Density-Aware Probabilistic Matching](https://mlsanity.com/paper/2609.16149/) (2026) ## Abstract (arXiv) Many existing works have made great strides towards reducing racial bias in face recognition. However, most of these methods attempt to rectify bias that manifests in models during training instead of directly addressing a major source of the bias, the dataset itself. Exceptions to this are BUPT-Balancedface/RFW and Fairface, but these works assume that primarily training on a single race or not racially balancing the dataset are inherently disadvantageous. We demonstrate that these assumptions are not necessarily valid. In our experiments, training on only African faces induced less bias than training on a balanced distribution of faces and distributions skewed to include more African faces produced more equitable models. We additionally notice that adding more images of existing identities to a dataset in place of adding new identities can lead to accuracy boosts across racial categories. Our code is available at https://github.com/j-alex-hanson/rethinking-race-face-datasets.