# Understanding Unequal Gender Classification Accuracy from Face Images

arXiv: https://arxiv.org/abs/1812.00099 · HTML: https://mlsanity.com/paper/1812.00099/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2018.

## Built on by (most cited first, 25 of 25)

- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021)
- [Discover the Unknown Biased Attribute of an Image Classifier](https://mlsanity.com/paper/2104.14556/) (2021)
- [Gendered Differences in Face Recognition Accuracy Explained by Hairstyles, Makeup, and Facial Morphology](https://mlsanity.com/paper/2112.14656/) (2021)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [Fair Classification with Adversarial Perturbations](https://mlsanity.com/paper/2106.05964/) (2021)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)
- [Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applications](https://mlsanity.com/paper/2104.14685/) (2021)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Verifiable evaluations of machine learning models using zkSNARKs](https://mlsanity.com/paper/2402.02675/) (2024)
- [Reliability and Validity of Image-Based and Self-Reported Skin Phenotype Metrics](https://mlsanity.com/paper/2106.11240/) (2021)
- [Simplicity Bias Leads to Amplified Performance Disparities](https://mlsanity.com/paper/2212.06641/) (2022)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Cascaded Debiasing: Studying the Cumulative Effect of Multiple Fairness-Enhancing Interventions](https://mlsanity.com/paper/2202.03734/) (2022)
- [Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness](https://mlsanity.com/paper/2211.05809/) (2022)
- [Matched sample selection with GANs for mitigating attribute confounding](https://mlsanity.com/paper/2103.13455/) (2021)
- [Does Face Recognition Error Echo Gender Classification Error?](https://mlsanity.com/paper/2104.13803/) (2021)
- [When Algorithms Play Favorites: Lookism in the Generation and Perception of Faces](https://mlsanity.com/paper/2506.11025/) (2025)
- [Freeze and Reveal: Exposing Modality Bias in Vision-Language Models](https://mlsanity.com/paper/2508.07432/) (2025)
- [Unifying Formal Explanations: A Complexity-Theoretic Perspective](https://mlsanity.com/paper/2602.18160/) (2026)
- [Towards Fair and Explainable AI using a Human-Centered AI Approach](https://mlsanity.com/paper/2306.07427/) (2023)
- ["My Kind of Woman": Analysing Gender Stereotypes in AI through The Averageness Theory and EU Law](https://mlsanity.com/paper/2407.17474/) (2024)
- [Facial Analysis Systems and Down Syndrome](https://mlsanity.com/paper/2502.06341/) (2025)
- [The Folly of AI for Age Verification](https://mlsanity.com/paper/2506.00038/) (2025)

## Abstract (arXiv)

Recent work shows unequal performance of commercial face classification services in the gender classification task across intersectional groups defined by skin type and gender. Accuracy on dark-skinned females is significantly worse than on any other group. In this paper, we conduct several analyses to try to uncover the reason for this gap. The main finding, perhaps surprisingly, is that skin type is not the driver. This conclusion is reached via stability experiments that vary an image's skin type via color-theoretic methods, namely luminance mode-shift and optimal transport. A second suspect, hair length, is also shown not to be the driver via experiments on face images cropped to exclude the hair. Finally, using contrastive post-hoc explanation techniques for neural networks, we bring forth evidence suggesting that differences in lip, eye and cheek structure across ethnicity lead to the differences. Further, lip and eye makeup are seen as strong predictors for a female face, which is a troubling propagation of a gender stereotype.