# Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?

arXiv: https://arxiv.org/abs/1912.07398 · HTML: https://mlsanity.com/paper/1912.07398/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 30 of 56)

- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [The Political Preferences of LLMs](https://mlsanity.com/paper/2402.01789/) (2024)
- [Face Recognition Accuracy Across Demographics: Shining a Light Into the Problem](https://mlsanity.com/paper/2206.01881/) (2022)
- [PressureVision: Estimating Hand Pressure from a Single RGB Image](https://mlsanity.com/paper/2203.10385/) (2022)
- [An Axiomatic Theory of Provably-Fair Welfare-Centric Machine Learning](https://mlsanity.com/paper/2104.14504/) (2021)
- [Evaluating Proposed Fairness Models for Face Recognition Algorithms](https://mlsanity.com/paper/2203.05051/) (2022)
- [FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations](https://mlsanity.com/paper/2304.14252/) (2023)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Towards Fair Face Verification: An In-depth Analysis of Demographic Biases](https://mlsanity.com/paper/2307.10011/) (2023)
- [Fair SA: Sensitivity Analysis for Fairness in Face Recognition](https://mlsanity.com/paper/2202.03586/) (2022)
- [A Deep Dive into Dataset Imbalance and Bias in Face Identification](https://mlsanity.com/paper/2203.08235/) (2022)
- [Bias and Diversity in Synthetic-based Face Recognition](https://mlsanity.com/paper/2311.03970/) (2023)
- [Going Deeper Into Face Detection: A Survey](https://mlsanity.com/paper/2103.14983/) (2021)
- [ONOT: a High-Quality ICAO-compliant Synthetic Mugshot Dataset](https://mlsanity.com/paper/2404.11236/) (2024)
- [CASIA-Face-Africa: A Large-scale African Face Image Database](https://mlsanity.com/paper/2105.03632/) (2021)
- [VisCUIT: Visual Auditor for Bias in CNN Image Classifier](https://mlsanity.com/paper/2204.05899/) (2022)
- [Hierarchical Attention-based Age Estimation and Bias Estimation](https://mlsanity.com/paper/2103.09882/) (2021)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021)
- [Does lossy image compression affect racial bias within face recognition?](https://mlsanity.com/paper/2208.07613/) (2022)
- [Assessing Bias in Face Image Quality Assessment](https://mlsanity.com/paper/2211.15265/) (2022)
- [Optimization-Based Improvement of Face Image Quality Assessment Techniques](https://mlsanity.com/paper/2305.14856/) (2023)
- [Beard Segmentation and Recognition Bias](https://mlsanity.com/paper/2308.15740/) (2023)
- [Can the accuracy bias by facial hairstyle be reduced through balancing the training data?](https://mlsanity.com/paper/2405.20062/) (2024)
- [Matched sample selection with GANs for mitigating attribute confounding](https://mlsanity.com/paper/2103.13455/) (2021)
- [Learning Fair Face Representation With Progressive Cross Transformer](https://mlsanity.com/paper/2108.04983/) (2021)
- [Twin identification over viewpoint change: A deep convolutional neural network surpasses humans](https://mlsanity.com/paper/2207.05316/) (2022)
- [Score Normalization for Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2407.14087/) (2024)
- [A large-scale study of performance and equity of commercial remote identity verification technologies across demographics](https://mlsanity.com/paper/2409.12318/) (2024)

## Abstract (arXiv)

Previous generations of face recognition algorithms differ in accuracy for images of different races (race bias). Here, we present the possible underlying factors (data-driven and scenario modeling) and methodological considerations for assessing race bias in algorithms. We discuss data driven factors (e.g., image quality, image population statistics, and algorithm architecture), and scenario modeling factors that consider the role of the "user" of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate how these issues apply, we present data from four face recognition algorithms (a previous-generation algorithm and three deep convolutional neural networks, DCNNs) for East Asian and Caucasian faces. First, dataset difficulty affected both overall recognition accuracy and race bias, such that race bias increased with item difficulty. Second, for all four algorithms, the degree of bias varied depending on the identification decision threshold. To achieve equal false accept rates (FARs), East Asian faces required higher identification thresholds than Caucasian faces, for all algorithms. Third, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude that race bias needs to be measured for individual applications and we provide a checklist for measuring this bias in face recognition algorithms.