# FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition

arXiv: https://arxiv.org/abs/2009.07838 · HTML: https://mlsanity.com/paper/2009.07838/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 26 of 26)

- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Consistent Instance False Positive Improves Fairness in Face Recognition](https://mlsanity.com/paper/2106.05519/) (2021)
- [Pre-training strategies and datasets for facial representation learning](https://mlsanity.com/paper/2103.16554/) (2021)
- [FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations](https://mlsanity.com/paper/2304.14252/) (2023)
- [Balancing Biases and Preserving Privacy on Balanced Faces in the Wild](https://mlsanity.com/paper/2103.09118/) (2021)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Fair SA: Sensitivity Analysis for Fairness in Face Recognition](https://mlsanity.com/paper/2202.03586/) (2022)
- [Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets](https://mlsanity.com/paper/2305.09072/) (2023)
- [Racial Bias within Face Recognition: A Survey](https://mlsanity.com/paper/2305.00817/) (2023)
- [Two-Face: Adversarial Audit of Commercial Face Recognition Systems](https://mlsanity.com/paper/2111.09137/) (2021)
- [Review of Demographic Fairness in Face Recognition](https://mlsanity.com/paper/2502.02309/) (2025)
- [Explaining Bias in Deep Face Recognition via Image Characteristics](https://mlsanity.com/paper/2208.11099/) (2022)
- [The Box Size Confidence Bias Harms Your Object Detector](https://mlsanity.com/paper/2112.01901/) (2021)
- [Does lossy image compression affect racial bias within face recognition?](https://mlsanity.com/paper/2208.07613/) (2022)
- [BAdd: Bias Mitigation through Bias Addition](https://mlsanity.com/paper/2408.11439/) (2024)
- [Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications](https://mlsanity.com/paper/2510.21887/) (2025)
- [Longitudinal Analysis of Mask and No-Mask on Child Face Recognition](https://mlsanity.com/paper/2111.00121/) (2021)
- [Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations](https://mlsanity.com/paper/2311.18575/) (2023)
- [Prototype Memory for Large-scale Face Representation Learning](https://mlsanity.com/paper/2105.02103/) (2021)
- [Color Invariant Skin Segmentation](https://mlsanity.com/paper/2204.09882/) (2022)
- [Inclusive Portraits: Race-Aware Human-in-the-Loop Technology](https://mlsanity.com/paper/2311.03567/) (2023)
- [The More Secure, The Less Equally Usable: Gender and Ethnicity (Un)fairness of Deep Face Recognition along Security Thresholds](https://mlsanity.com/paper/2209.15550/) (2022)
- [Test-Time Augmentation for Pose-invariant Face Recognition](https://mlsanity.com/paper/2505.09256/) (2025)
- [SELM: Siamese Extreme Learning Machine with Application to Face Biometrics](https://mlsanity.com/paper/2108.03140/) (2021)
- [Mask-up: Investigating Biases in Face Re-identification for Masked Faces](https://mlsanity.com/paper/2402.13771/) (2024)
- [Mitigating Bias in Facial Recognition Systems: Centroid Fairness Loss Optimization](https://mlsanity.com/paper/2504.19370/) (2025)

## Abstract (arXiv)

This work summarizes the 2020 ChaLearn Looking at People Fair Face Recognition and Analysis Challenge and provides a description of the top-winning solutions and analysis of the results. The aim of the challenge was to evaluate accuracy and bias in gender and skin colour of submitted algorithms on the task of 1:1 face verification in the presence of other confounding attributes. Participants were evaluated using an in-the-wild dataset based on reannotated IJB-C, further enriched by 12.5K new images and additional labels. The dataset is not balanced, which simulates a real world scenario where AI-based models supposed to present fair outcomes are trained and evaluated on imbalanced data. The challenge attracted 151 participants, who made more than 1.8K submissions in total. The final phase of the challenge attracted 36 active teams out of which 10 exceeded 0.999 AUC-ROC while achieving very low scores in the proposed bias metrics. Common strategies by the participants were face pre-processing, homogenization of data distributions, the use of bias aware loss functions and ensemble models. The analysis of top-10 teams shows higher false positive rates (and lower false negative rates) for females with dark skin tone as well as the potential of eyeglasses and young age to increase the false positive rates too.