# Characterizing the Variability in Face Recognition Accuracy Relative to Race

arXiv: https://arxiv.org/abs/1904.07325 · HTML: https://mlsanity.com/paper/1904.07325/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2019.

## Built on by (most cited first, 30 of 40)

- [Meta Balanced Network for Fair Face Recognition](https://mlsanity.com/paper/2205.06548/) (2022)
- [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021)
- [Analyzing Bias in Diffusion-based Face Generation Models](https://mlsanity.com/paper/2305.06402/) (2023)
- [One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision](https://mlsanity.com/paper/2102.02320/) (2021)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [About Face: A Survey of Facial Recognition Evaluation](https://mlsanity.com/paper/2102.00813/) (2021)
- [UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models](https://mlsanity.com/paper/2110.15499/) (2021)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Ethical Considerations for Responsible Data Curation](https://mlsanity.com/paper/2302.03629/) (2023)
- [Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applications](https://mlsanity.com/paper/2104.14685/) (2021)
- [Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics](https://mlsanity.com/paper/2106.11240/) (2021)
- [The Gender Gap in Face Recognition Accuracy Is a Hairy Problem](https://mlsanity.com/paper/2206.04867/) (2022)
- [50 Years of Automated Face Recognition](https://mlsanity.com/paper/2505.24247/) (2025)
- [Fairness in Face Presentation Attack Detection](https://mlsanity.com/paper/2209.09035/) (2022)
- [What Should Be Balanced in a "Balanced" Face Recognition Dataset?](https://mlsanity.com/paper/2304.09818/) (2023)
- [Fair SA: Sensitivity Analysis for Fairness in Face Recognition](https://mlsanity.com/paper/2202.03586/) (2022)
- [An Examination of Bias of Facial Analysis based BMI Prediction Models](https://mlsanity.com/paper/2204.10262/) (2022)
- [Causality-Inspired Taxonomy for Explainable Artificial Intelligence](https://mlsanity.com/paper/2208.09500/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021)
- [Information-Theoretic Bias Assessment Of Learned Representations Of Pretrained Face Recognition](https://mlsanity.com/paper/2111.04673/) (2021)
- [Impact of Blur and Resolution on Demographic Disparities in 1-to-Many Facial Identification](https://mlsanity.com/paper/2309.04447/) (2023)
- [Matched sample selection with GANs for mitigating attribute confounding](https://mlsanity.com/paper/2103.13455/) (2021)
- [Does Face Recognition Error Echo Gender Classification Error?](https://mlsanity.com/paper/2104.13803/) (2021)
- [NOMAD: A Natural, Occluded, Multi-scale Aerial Dataset, for Emergency Response Scenarios](https://mlsanity.com/paper/2309.09518/) (2023)
- [Score Normalization for Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2407.14087/) (2024)
- [Lights, Camera, Matching: The Role of Image Illumination in Fair Face Recognition](https://mlsanity.com/paper/2501.08910/) (2025)
- [Exploring Causes of Demographic Variations In Face Recognition Accuracy](https://mlsanity.com/paper/2304.07175/) (2023)
- [Confidence Intervals for Error Rates in 1:1 Matching Tasks: Critical Statistical Analysis and Recommendations](https://mlsanity.com/paper/2306.01198/) (2023)

## Abstract (arXiv)

Many recent news headlines have labeled face recognition technology as biased or racist. We report on a methodical investigation into differences in face recognition accuracy between African-American and Caucasian image cohorts of the MORPH dataset. We find that, for all four matchers considered, the impostor and the genuine distributions are statistically significantly different between cohorts. For a fixed decision threshold, the African-American image cohort has a higher false match rate and a lower false non-match rate. ROC curves compare verification rates at the same false match rate, but the different cohorts achieve the same false match rate at different thresholds. This means that ROC comparisons are not relevant to operational scenarios that use a fixed decision threshold. We show that, for the ResNet matcher, the two cohorts have approximately equal separation of impostor and genuine distributions. Using ICAO compliance as a standard of image quality, we find that the initial image cohorts have unequal rates of good quality images. The ICAO-compliant subsets of the original image cohorts show improved accuracy, with the main effect being to reducing the low-similarity tail of the genuine distributions.