# Explainable Face Recognition

arXiv: https://arxiv.org/abs/2008.00916 · HTML: https://mlsanity.com/paper/2008.00916/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2020.

## Built on by (most cited first, 30 of 34)

- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [TopicFM: Robust and Interpretable Topic-Assisted Feature Matching](https://mlsanity.com/paper/2207.00328/) (2022)
- [Towards Interpretable Deep Metric Learning with Structural Matching](https://mlsanity.com/paper/2108.05889/) (2021)
- [Inroads into Autonomous Network Defence using Explained Reinforcement Learning](https://mlsanity.com/paper/2306.09318/) (2023)
- [Pixel-Level Face Image Quality Assessment for Explainable Face Recognition](https://mlsanity.com/paper/2110.11001/) (2021)
- [Towards Visual Saliency Explanations of Face Verification](https://mlsanity.com/paper/2305.08546/) (2023)
- [TopicFM+: Boosting Accuracy and Efficiency of Topic-Assisted Feature Matching](https://mlsanity.com/paper/2307.00485/) (2023)
- [Causality-Inspired Taxonomy for Explainable Artificial Intelligence](https://mlsanity.com/paper/2208.09500/) (2022)
- [HP-Capsule: Unsupervised Face Part Discovery by Hierarchical Parsing Capsule Network](https://mlsanity.com/paper/2203.10699/) (2022)
- [Explainable Diabetic Retinopathy Detection and Retinal Image Generation](https://mlsanity.com/paper/2107.00296/) (2021)
- [Explainable Face Verification via Feature-Guided Gradient Backpropagation](https://mlsanity.com/paper/2403.04549/) (2024)
- [Interpreting Face Inference Models using Hierarchical Network Dissection](https://mlsanity.com/paper/2108.10360/) (2021)
- [Precise and Generalized Robustness Certification for Neural Networks](https://mlsanity.com/paper/2306.06747/) (2023)
- [Explainable Artificial Intelligence to Detect Image Spam Using Convolutional Neural Network](https://mlsanity.com/paper/2209.03166/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images](https://mlsanity.com/paper/2303.10896/) (2023)
- [Explanation of Face Recognition via Saliency Maps](https://mlsanity.com/paper/2304.06118/) (2023)
- [Discriminative Deep Feature Visualization for Explainable Face Recognition](https://mlsanity.com/paper/2306.00402/) (2023)
- [Face-GPS: A Comprehensive Technique for Quantifying Facial Muscle Dynamics in Videos](https://mlsanity.com/paper/2401.05625/) (2024)
- [Explaining Face Presentation Attack Detection Using Natural Language](https://mlsanity.com/paper/2111.04862/) (2021)
- [Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN](https://mlsanity.com/paper/2201.09689/) (2022)
- [Multi-Depth Concept Extraction for Post-Hoc Vision Encoder Explanation](https://mlsanity.com/paper/2411.19700/) (2024)
- [VerLM: Explaining Face Verification Using Natural Language](https://mlsanity.com/paper/2601.01798/) (2026)
- [MLLM-based Textual Explanations for Face Comparison](https://mlsanity.com/paper/2603.16629/) (2026)
- [Bridging the Gap: Gaze Events as Interpretable Concepts to Explain Deep Neural Sequence Models](https://mlsanity.com/paper/2304.13536/) (2023)
- [Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systems](https://mlsanity.com/paper/2407.05983/) (2024)
- [Explainable Face Recognition via Improved Localization](https://mlsanity.com/paper/2505.03837/) (2025)
- [Explaining Deep Face Algorithms through Visualization: A Survey](https://mlsanity.com/paper/2309.14715/) (2023)
- [Explainable offline automatic signature verifier to support forensic handwriting examiners](https://mlsanity.com/paper/2405.12695/) (2024)
- [Leveraging Activations for Superpixel Explanations](https://mlsanity.com/paper/2406.04933/) (2024)

## Abstract (arXiv)

Explainable face recognition is the problem of explaining why a facial matcher matches faces. In this paper, we provide the first comprehensive benchmark and baseline evaluation for explainable face recognition. We define a new evaluation protocol called the ``inpainting game'', which is a curated set of 3648 triplets (probe, mate, nonmate) of 95 subjects, which differ by synthetically inpainting a chosen facial characteristic like the nose, eyebrows or mouth creating an inpainted nonmate. An explainable face matcher is tasked with generating a network attention map which best explains which regions in a probe image match with a mated image, and not with an inpainted nonmate for each triplet. This provides ground truth for quantifying what image regions contribute to face matching. Furthermore, we provide a comprehensive benchmark on this dataset comparing five state of the art methods for network attention in face recognition on three facial matchers. This benchmark includes two new algorithms for network attention called subtree EBP and Density-based Input Sampling for Explanation (DISE) which outperform the state of the art by a wide margin. Finally, we show qualitative visualization of these network attention techniques on novel images, and explore how these explainable face recognition models can improve transparency and trust for facial matchers.