# Fair SA: Sensitivity Analysis for Fairness in Face Recognition

arXiv: https://arxiv.org/abs/2202.03586 · HTML: https://mlsanity.com/paper/2202.03586/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 8 Feb 2022.

## Builds on (most cited first, 16 of 16)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Fairness in Deep Learning: A Computational Perspective](https://mlsanity.com/paper/1908.08843/) (2019)
- [Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing](https://mlsanity.com/paper/2001.00964/) (2020)
- [Accuracy comparison across face recognition algorithms: Where are we on measuring race bias?](https://mlsanity.com/paper/1912.07398/) (2019)
- [Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning](https://mlsanity.com/paper/2006.12621/) (2020)
- [Characterizing the Variability in Face Recognition Accuracy Relative to Race](https://mlsanity.com/paper/1904.07325/) (2019)
- [FairFace Challenge at ECCV 2020: Analyzing Bias in Face Recognition](https://mlsanity.com/paper/2009.07838/) (2020)
- [How Image Degradations Affect Deep CNN-based Face Recognition?](https://mlsanity.com/paper/1608.05246/) (2016)
- [Strengths and Weaknesses of Deep Learning Models for Face Recognition Against Image Degradations](https://mlsanity.com/paper/1710.01494/) (2017)
- [Visual Psychophysics for Making Face Recognition Algorithms More Explainable](https://mlsanity.com/paper/1803.07140/) (2018)
- [PsyPhy: A Psychophysics Driven Evaluation Framework for Visual Recognition](https://mlsanity.com/paper/1611.06448/) (2016)
- [Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1801.08116/) (2018)
- [FACESEC: A Fine-grained Robustness Evaluation Framework for Face Recognition Systems](https://mlsanity.com/paper/2104.04107/) (2021)
- [Recognizing Disguised Faces in the Wild](https://mlsanity.com/paper/1811.08837/) (2018)
- [SeesawFaceNets: sparse and robust face verification model for mobile platform](https://mlsanity.com/paper/1908.09124/) (2019)

## Built on by (most cited first, 11 of 11)

- [TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models](https://mlsanity.com/paper/2312.01261/) (2023)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [Does lossy image compression affect racial bias within face recognition?](https://mlsanity.com/paper/2208.07613/) (2022)
- [Class Distribution Shifts in Zero-Shot Learning: Learning Robust Representations](https://mlsanity.com/paper/2311.18575/) (2023)
- [Designing Data: Proactive Data Collection and Iteration for Machine Learning](https://mlsanity.com/paper/2301.10319/) (2023)
- [UnbiasedNets: A Dataset Diversification Framework for Robustness Bias Alleviation in Neural Networks](https://mlsanity.com/paper/2302.12538/) (2023)
- [LabellessFace: Fair Metric Learning for Face Recognition without Attribute Labels](https://mlsanity.com/paper/2409.09274/) (2024)
- [BiasConnect: Investigating Bias Interactions in Text-to-Image Models](https://mlsanity.com/paper/2503.09763/) (2025)
- [Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models](https://mlsanity.com/paper/2505.17280/) (2025)
- [Investigating Bias and Fairness in Appearance-based Gaze Estimation](https://mlsanity.com/paper/2604.10707/) (2026)
- [Utilizing Adversarial Examples for Bias Mitigation and Accuracy Enhancement](https://mlsanity.com/paper/2404.11819/) (2024)

## Abstract (arXiv)

As the use of deep learning in high impact domains becomes ubiquitous, it is increasingly important to assess the resilience of models. One such high impact domain is that of face recognition, with real world applications involving images affected by various degradations, such as motion blur or high exposure. Moreover, images captured across different attributes, such as gender and race, can also challenge the robustness of a face recognition algorithm. While traditional summary statistics suggest that the aggregate performance of face recognition models has continued to improve, these metrics do not directly measure the robustness or fairness of the models. Visual Psychophysics Sensitivity Analysis (VPSA) [1] provides a way to pinpoint the individual causes of failure by way of introducing incremental perturbations in the data. However, perturbations may affect subgroups differently. In this paper, we propose a new fairness evaluation based on robustness in the form of a generic framework that extends VPSA. With this framework, we can analyze the ability of a model to perform fairly for different subgroups of a population affected by perturbations, and pinpoint the exact failure modes for a subgroup by measuring targeted robustness. With the increasing focus on the fairness of models, we use face recognition as an example application of our framework and propose to compactly visualize the fairness analysis of a model via AUC matrices. We analyze the performance of common face recognition models and empirically show that certain subgroups are at a disadvantage when images are perturbed, thereby uncovering trends that were not visible using the model's performance on subgroups without perturbations.