# The Gender Gap in Face Recognition Accuracy Is a Hairy Problem

arXiv: https://arxiv.org/abs/2206.04867 · HTML: https://mlsanity.com/paper/2206.04867/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019)
- [img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation](https://mlsanity.com/paper/2012.07791/) (2020)
- [Analysis of Gender Inequality In Face Recognition Accuracy](https://mlsanity.com/paper/2002.00065/) (2020)
- [Characterizing the Variability in Face Recognition Accuracy Relative to Race](https://mlsanity.com/paper/1904.07325/) (2019)
- [How Does Gender Balance In Training Data Affect Face Recognition Accuracy?](https://mlsanity.com/paper/2002.02934/) (2020)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [An Experimental Evaluation of Covariates Effects on Unconstrained Face Verification](https://mlsanity.com/paper/1808.05508/) (2018)
- [Is Face Recognition Sexist? No, Gendered Hairstyles and Biology Are](https://mlsanity.com/paper/2008.06989/) (2020)
- [Does Face Recognition Accuracy Get Better With Age? Deep Face Matchers Say No](https://mlsanity.com/paper/1911.06396/) (2019)

## Built on by (most cited first, 15 of 15)

- [Logical Consistency and Greater Descriptive Power for Facial Hair Attribute Learning](https://mlsanity.com/paper/2302.11102/) (2023)
- [What Should Be Balanced in a "Balanced" Face Recognition Dataset?](https://mlsanity.com/paper/2304.09818/) (2023)
- [Bias and Diversity in Synthetic-based Face Recognition](https://mlsanity.com/paper/2311.03970/) (2023)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Beard Segmentation and Recognition Bias](https://mlsanity.com/paper/2308.15740/) (2023)
- [Impact of Blur and Resolution on Demographic Disparities in 1-to-Many Facial Identification](https://mlsanity.com/paper/2309.04447/) (2023)
- [Can the accuracy bias by facial hairstyle be reduced through balancing the training data?](https://mlsanity.com/paper/2405.20062/) (2024)
- [LogicNet: A Logical Consistency Embedded Face Attribute Learning Network](https://mlsanity.com/paper/2311.11208/) (2023)
- [Impact of Sunglasses on One-to-Many Facial Identification Accuracy](https://mlsanity.com/paper/2412.05721/) (2024)
- [Lights, Camera, Matching: The Role of Image Illumination in Fair Face Recognition](https://mlsanity.com/paper/2501.08910/) (2025)
- [CounterFace: A Synthetic Face Dataset for Fine-Grained Counterfactual Evaluation of Face Recognition Systems](https://mlsanity.com/paper/2407.13922/) (2024)
- [On the Illusion of Gender Bias in Face Recognition: Explaining the Fairness Issue Through Non-demographic Attributes](https://mlsanity.com/paper/2501.12020/) (2025)
- [What's color got to do with it? Face recognition in grayscale](https://mlsanity.com/paper/2309.05180/) (2023)
- [Vec2Face+ for Face Dataset Generation](https://mlsanity.com/paper/2507.17192/) (2025)
- [Demographic Fairness in Multimodal LLMs: A Benchmark of Gender and Ethnicity Bias in Face Verification](https://mlsanity.com/paper/2603.25613/) (2026)

## Abstract (arXiv)

It is broadly accepted that there is a "gender gap" in face recognition accuracy, with females having higher false match and false non-match rates. However, relatively little is known about the cause(s) of this gender gap. Even the recent NIST report on demographic effects lists "analyze cause and effect" under "what we did not do". We first demonstrate that female and male hairstyles have important differences that impact face recognition accuracy. In particular, compared to females, male facial hair contributes to creating a greater average difference in appearance between different male faces. We then demonstrate that when the data used to estimate recognition accuracy is balanced across gender for how hairstyles occlude the face, the initially observed gender gap in accuracy largely disappears. We show this result for two different matchers, and analyzing images of Caucasians and of African-Americans. These results suggest that future research on demographic variation in accuracy should include a check for balanced quality of the test data as part of the problem formulation. To promote reproducible research, matchers, attribute classifiers, and datasets used in this research are/will be publicly available.