# Mask-up: Investigating Biases in Face Re-identification for Masked Faces

arXiv: https://arxiv.org/abs/2402.13771 · HTML: https://mlsanity.com/paper/2402.13771/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 101 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Deep Learning Face Representation by Joint Identification-Verification](https://mlsanity.com/paper/1406.4773/) (2014)
- [Demographic Bias in Biometrics: A Survey on an Emerging Challenge](https://mlsanity.com/paper/2003.02488/) (2020)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [Sample and Computation Redistribution for Efficient Face Detection](https://mlsanity.com/paper/2105.04714/) (2021)
- [Face Recognition: Too Bias, or Not Too Bias?](https://mlsanity.com/paper/2002.06483/) (2020)
- [Towards Measuring Fairness in AI: the Casual Conversations Dataset](https://mlsanity.com/paper/2104.02821/) (2021)
- [SFace: Privacy-friendly and Accurate Face Recognition using Synthetic Data](https://mlsanity.com/paper/2206.10520/) (2022)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)
- [Masked Face Recognition for Secure Authentication](https://mlsanity.com/paper/2008.11104/) (2020)
- [Characterizing Intersectional Group Fairness with Worst-Case Comparisons](https://mlsanity.com/paper/2101.01673/) (2021)
- [Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective](https://mlsanity.com/paper/1703.09471/) (2017)
- [A survey of face recognition techniques under occlusion](https://mlsanity.com/paper/2006.11366/) (2020)
- [Empirically Analyzing the Effect of Dataset Biases on Deep Face Recognition Systems](https://mlsanity.com/paper/1712.01619/) (2017)
- [About Face: A Survey of Facial Recognition Evaluation](https://mlsanity.com/paper/2102.00813/) (2021)
- [FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition](https://mlsanity.com/paper/2009.07838/) (2020)
- [Deep Learning for Face Recognition: Pride or Prejudiced?](https://mlsanity.com/paper/1904.01219/) (2019)
- [Face-Off: Adversarial Face Obfuscation](https://mlsanity.com/paper/2003.08861/) (2020)
- [End2End Occluded Face Recognition by Masking Corrupted Features](https://mlsanity.com/paper/2108.09468/) (2021)
- [Human-In-The-Loop Person Re-Identification](https://mlsanity.com/paper/1612.01345/) (2016)
- [Adversarial Attacks on Face Detectors using Neural Net based Constrained Optimization](https://mlsanity.com/paper/1805.12302/) (2018)
- [Grid Loss: Detecting Occluded Faces](https://mlsanity.com/paper/1609.00129/) (2016)
- [On the Robustness of Face Recognition Algorithms Against Attacks and Bias](https://mlsanity.com/paper/2002.02942/) (2020)
- [Measuring Hidden Bias within Face Recognition via Racial Phenotypes](https://mlsanity.com/paper/2110.09839/) (2021)
- [Detection of Face Recognition Adversarial Attacks](https://mlsanity.com/paper/1912.02918/) (2019)
- [Two-Face: Adversarial Audit of Commercial Face Recognition Systems](https://mlsanity.com/paper/2111.09137/) (2021)
- [Robustness Disparities in Face Detection](https://mlsanity.com/paper/2211.15937/) (2022)
- [Indian Masked Faces in the Wild Dataset](https://mlsanity.com/paper/2106.09670/) (2021)
- [Balanced Masked and Standard Face Recognition](https://mlsanity.com/paper/2110.01521/) (2021)
- [Cinderella's shoe won't fit Soundarya: An audit of facial processing tools on Indian faces](https://mlsanity.com/paper/2112.09326/) (2021)

## Abstract (arXiv)

AI based Face Recognition Systems (FRSs) are now widely distributed and deployed as MLaaS solutions all over the world, moreso since the COVID-19 pandemic for tasks ranging from validating individuals' faces while buying SIM cards to surveillance of citizens. Extensive biases have been reported against marginalized groups in these systems and have led to highly discriminatory outcomes. The post-pandemic world has normalized wearing face masks but FRSs have not kept up with the changing times. As a result, these systems are susceptible to mask based face occlusion. In this study, we audit four commercial and nine open-source FRSs for the task of face re-identification between different varieties of masked and unmasked images across five benchmark datasets (total 14,722 images). These simulate a realistic validation/surveillance task as deployed in all major countries around the world. Three of the commercial and five of the open-source FRSs are highly inaccurate; they further perpetuate biases against non-White individuals, with the lowest accuracy being 0%. A survey for the same task with 85 human participants also results in a low accuracy of 40%. Thus a human-in-the-loop moderation in the pipeline does not alleviate the concerns, as has been frequently hypothesized in literature. Our large-scale study shows that developers, lawmakers and users of such services need to rethink the design principles behind FRSs, especially for the task of face re-identification, taking cognizance of observed biases.