# Measuring Hidden Bias within Face Recognition via Racial Phenotypes arXiv: https://arxiv.org/abs/2110.09839 ยท HTML: https://mlsanity.com/paper/2110.09839/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 55 references that are in the map. - First posted on arXiv 19 Oct 2021. ## Builds on (most cited first, 20 of 20) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017) - [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017) - [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016) - [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017) - [Mitigate Bias in Face Recognition using Skewness-Aware Reinforcement Learning](https://mlsanity.com/paper/1911.10692/) (2019) - [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020) - [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019) - [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021) - [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020) - [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019) - [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019) - [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021) - [MAAD-Face: A Massively Annotated Attribute Dataset for Face Images](https://mlsanity.com/paper/2012.01030/) (2020) - [Diversity and Inclusion Metrics in Subset Selection](https://mlsanity.com/paper/2002.03256/) (2020) - [FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition](https://mlsanity.com/paper/2009.07838/) (2020) - [Understanding Unequal Gender Classification Accuracy from Face Images](https://mlsanity.com/paper/1812.00099/) (2018) - [Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applications](https://mlsanity.com/paper/2104.14685/) (2021) - [Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics](https://mlsanity.com/paper/2106.11240/) (2021) - [Naive-Deep Face Recognition: Touching the Limit of LFW Benchmark or Not?](https://mlsanity.com/paper/1501.04690/) (2015) ## Built on by (most cited first, 16 of 16) - [Arc2Face: A Foundation Model for ID-Consistent Human Faces](https://mlsanity.com/paper/2403.11641/) (2024) - [Ethical Considerations for Responsible Data Curation](https://mlsanity.com/paper/2302.03629/) (2023) - [Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets](https://mlsanity.com/paper/2305.09072/) (2023) - [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023) - [Does lossy image compression affect racial bias within face recognition?](https://mlsanity.com/paper/2208.07613/) (2022) - [ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling](https://mlsanity.com/paper/2405.16570/) (2024) - [Can the accuracy bias by facial hairstyle be reduced through balancing the training data?](https://mlsanity.com/paper/2405.20062/) (2024) - [A large-scale study of performance and equity of commercial remote identity verification technologies across demographics](https://mlsanity.com/paper/2409.12318/) (2024) - [Improved HER2 Tumor Segmentation with Subtype Balancing using Deep Generative Networks](https://mlsanity.com/paper/2211.06150/) (2022) - [Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification](https://mlsanity.com/paper/2412.03349/) (2024) - [FaceX: Understanding Face Attribute Classifiers through Summary Model Explanations](https://mlsanity.com/paper/2412.07313/) (2024) - [Disentangling Racial Phenotypes: Fine-Grained Control of Race-related Facial Phenotype Characteristics](https://mlsanity.com/paper/2403.19897/) (2024) - [Gone With the Bits: Revealing Racial Bias in Low-Rate Neural Compression for Facial Images](https://mlsanity.com/paper/2505.02949/) (2025) - [Mask-up: Investigating Biases in Face Re-identification for Masked Faces](https://mlsanity.com/paper/2402.13771/) (2024) - [Component-Based Fairness in Face Attribute Classification with Bayesian Network-informed Meta Learning](https://mlsanity.com/paper/2505.01699/) (2025) - [Position: Fairness Failure in Generative Models is an Evaluation Problem](https://mlsanity.com/paper/2608.16974/) (2026) ## Abstract (arXiv) Recent work reports disparate performance for intersectional racial groups across face recognition tasks: face verification and identification. However, the definition of those racial groups has a significant impact on the underlying findings of such racial bias analysis. Previous studies define these groups based on either demographic information (e.g. African, Asian etc.) or skin tone (e.g. lighter or darker skins). The use of such sensitive or broad group definitions has disadvantages for bias investigation and subsequent counter-bias solutions design. By contrast, this study introduces an alternative racial bias analysis methodology via facial phenotype attributes for face recognition. We use the set of observable characteristics of an individual face where a race-related facial phenotype is hence specific to the human face and correlated to the racial profile of the subject. We propose categorical test cases to investigate the individual influence of those attributes on bias within face recognition tasks. We compare our phenotype-based grouping methodology with previous grouping strategies and show that phenotype-based groupings uncover hidden bias without reliance upon any potentially protected attributes or ill-defined grouping strategies. Furthermore, we contribute corresponding phenotype attribute category labels for two face recognition tasks: RFW for face verification and VGGFace2 (test set) for face identification.