# Analysis of Gender Inequality In Face Recognition Accuracy

arXiv: https://arxiv.org/abs/2002.00065 · HTML: https://mlsanity.com/paper/2002.00065/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2020.

## Built on by (most cited first, 30 of 47)

- [Towards Measuring Fairness in AI: the Casual Conversations Dataset](https://mlsanity.com/paper/2104.02821/) (2021)
- [Towards Fair Federated Learning with Zero-Shot Data Augmentation](https://mlsanity.com/paper/2104.13417/) (2021)
- [Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers](https://mlsanity.com/paper/2203.04913/) (2022)
- [Discover and Mitigate Unknown Biases with Debiasing Alternate Networks](https://mlsanity.com/paper/2207.10077/) (2022)
- [PASS: Protected Attribute Suppression System for Mitigating Bias in Face Recognition](https://mlsanity.com/paper/2108.03764/) (2021)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [AI-Face: A Million-Scale Demographically Annotated AI-Generated Face Dataset and Fairness Benchmark](https://mlsanity.com/paper/2406.00783/) (2024)
- [Rethinking Common Assumptions to Mitigate Racial Bias in Face Recognition Datasets](https://mlsanity.com/paper/2109.03229/) (2021)
- [Anatomizing Bias in Facial Analysis](https://mlsanity.com/paper/2112.06522/) (2021)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)
- [The Gender Gap in Face Recognition Accuracy Is a Hairy Problem](https://mlsanity.com/paper/2206.04867/) (2022)
- [Fairness in Face Presentation Attack Detection](https://mlsanity.com/paper/2209.09035/) (2022)
- [What Should Be Balanced in a "Balanced" Face Recognition Dataset?](https://mlsanity.com/paper/2304.09818/) (2023)
- [A Deep Dive into Dataset Imbalance and Bias in Face Identification](https://mlsanity.com/paper/2203.08235/) (2022)
- [Going Deeper Into Face Detection: A Survey](https://mlsanity.com/paper/2103.14983/) (2021)
- [Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networks](https://mlsanity.com/paper/2204.04440/) (2022)
- [VisCUIT: Visual Auditor for Bias in CNN Image Classifier](https://mlsanity.com/paper/2204.05899/) (2022)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Distill and De-bias: Mitigating Bias in Face Verification using Knowledge Distillation](https://mlsanity.com/paper/2112.09786/) (2021)
- [Explaining Bias in Deep Face Recognition via Image Characteristics](https://mlsanity.com/paper/2208.11099/) (2022)
- [Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness](https://mlsanity.com/paper/2211.05809/) (2022)
- [Impact of Blur and Resolution on Demographic Disparities in 1-to-Many Facial Identification](https://mlsanity.com/paper/2309.04447/) (2023)
- [Does Face Recognition Error Echo Gender Classification Error?](https://mlsanity.com/paper/2104.13803/) (2021)
- [The Casual Conversations v2 Dataset](https://mlsanity.com/paper/2303.04838/) (2023)
- [Score Normalization for Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2407.14087/) (2024)
- [FineFACE: Fair Facial Attribute Classification Leveraging Fine-grained Features](https://mlsanity.com/paper/2408.16881/) (2024)
- [The Impact of Balancing Real and Synthetic Data on Accuracy and Fairness in Face Recognition](https://mlsanity.com/paper/2409.02867/) (2024)
- [The More Secure, The Less Equally Usable: Gender and Ethnicity (Un)fairness of Deep Face Recognition along Security Thresholds](https://mlsanity.com/paper/2209.15550/) (2022)
- [A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification](https://mlsanity.com/paper/2407.10104/) (2024)
- [CounterFace: A Synthetic Face Dataset for Fine-Grained Counterfactual Evaluation of Face Recognition Systems](https://mlsanity.com/paper/2407.13922/) (2024)

## Abstract (arXiv)

We present a comprehensive analysis of how and why face recognition accuracy differs between men and women. We show that accuracy is lower for women due to the combination of (1) the impostor distribution for women having a skew toward higher similarity scores, and (2) the genuine distribution for women having a skew toward lower similarity scores. We show that this phenomenon of the impostor and genuine distributions for women shifting closer towards each other is general across datasets of African-American, Caucasian, and Asian faces. We show that the distribution of facial expressions may differ between male/female, but that the accuracy difference persists for image subsets rated confidently as neutral expression. The accuracy difference also persists for image subsets rated as close to zero pitch angle. Even when removing images with forehead partially occluded by hair/hat, the same impostor/genuine accuracy difference persists. We show that the female genuine distribution improves when only female images without facial cosmetics are used, but that the female impostor distribution also degrades at the same time. Lastly, we show that the accuracy difference persists even if a state-of-the-art deep learning method is trained from scratch using training data explicitly balanced between male and female images and subjects.