# Are Explainability Tools Gender Biased? A Case Study on Face Presentation Attack Detection

arXiv: https://arxiv.org/abs/2304.13419 · HTML: https://mlsanity.com/paper/2304.13419/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [ElasticFace: Elastic Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/2109.09416/) (2021)
- [CelebA-Spoof: Large-Scale Face Anti-Spoofing Dataset with Rich Annotations](https://mlsanity.com/paper/2007.12342/) (2020)
- [Regularized Fine-grained Meta Face Anti-spoofing](https://mlsanity.com/paper/1911.10771/) (2019)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Revisiting Pixel-Wise Supervision for Face Anti-Spoofing](https://mlsanity.com/paper/2011.12032/) (2020)
- [Fairness in Biometrics: a figure of merit to assess biometric verification systems](https://mlsanity.com/paper/2011.02395/) (2020)
- [Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition](https://mlsanity.com/paper/2004.01019/) (2020)
- [Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechanism for Generalized Face Presentation Attack Detection](https://mlsanity.com/paper/2109.07950/) (2021)
- [SynthASpoof: Developing Face Presentation Attack Detection Based on Privacy-friendly Synthetic Data](https://mlsanity.com/paper/2303.02660/) (2023)
- [Pixel-Level Face Image Quality Assessment for Explainable Face Recognition](https://mlsanity.com/paper/2110.11001/) (2021)
- [Fairness in Face Presentation Attack Detection](https://mlsanity.com/paper/2209.09035/) (2022)
- [Causality-Inspired Taxonomy for Explainable Artificial Intelligence](https://mlsanity.com/paper/2208.09500/) (2022)
- [Myope Models -- Are face presentation attack detection models short-sighted?](https://mlsanity.com/paper/2111.11127/) (2021)
- [Assessing Bias in Face Image Quality Assessment](https://mlsanity.com/paper/2211.15265/) (2022)
- [Explaining Face Presentation Attack Detection Using Natural Language](https://mlsanity.com/paper/2111.04862/) (2021)

## Built on by (most cited first, 6 of 6)

- [Bias and Diversity in Synthetic-based Face Recognition](https://mlsanity.com/paper/2311.03970/) (2023)
- [Model Compression Techniques in Biometrics Applications: A Survey](https://mlsanity.com/paper/2401.10139/) (2024)
- [Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition](https://mlsanity.com/paper/2408.10175/) (2024)
- [Gender Representation and Bias in Indian Civil Service Mock Interviews](https://mlsanity.com/paper/2409.12194/) (2024)
- [Explanation-based Training with Differentiable Insertion/Deletion Metric-aware Regularizers](https://mlsanity.com/paper/2310.12553/) (2023)
- [Explainable Face Presentation Attack Detection via Ensemble-CAM](https://mlsanity.com/paper/2510.19695/) (2025)

## Abstract (arXiv)

Face recognition (FR) systems continue to spread in our daily lives with an increasing demand for higher explainability and interpretability of FR systems that are mainly based on deep learning. While bias across demographic groups in FR systems has already been studied, the bias of explainability tools has not yet been investigated. As such tools aim at steering further development and enabling a better understanding of computer vision problems, the possible existence of bias in their outcome can lead to a chain of biased decisions. In this paper, we explore the existence of bias in the outcome of explainability tools by investigating the use case of face presentation attack detection. By utilizing two different explainability tools on models with different levels of bias, we investigate the bias in the outcome of such tools. Our study shows that these tools show clear signs of gender bias in the quality of their explanations.