# Explainable Model-Agnostic Similarity and Confidence in Face Verification

arXiv: https://arxiv.org/abs/2211.13735 · HTML: https://mlsanity.com/paper/2211.13735/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 16 of 16)

- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Cross-Age LFW: A Database for Studying Cross-Age Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/1708.08197/) (2017)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Tell Me Where to Look: Guided Attention Inference Network](https://mlsanity.com/paper/1802.10171/) (2018)
- [Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs](https://mlsanity.com/paper/2008.02312/) (2020)
- [ElasticFace: Elastic Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/2109.09416/) (2021)
- [Face Transformer for Recognition](https://mlsanity.com/paper/2103.14803/) (2021)
- [Cross-Quality LFW: A Database for Analyzing Cross-Resolution Image Face Recognition in Unconstrained Environments](https://mlsanity.com/paper/2108.10290/) (2021)
- [xCos: An Explainable Cosine Metric for Face Verification Task](https://mlsanity.com/paper/2003.05383/) (2020)
- [Stating Comparison Score Uncertainty and Verification Decision Confidence Towards Transparent Face Recognition](https://mlsanity.com/paper/2210.10354/) (2022)
- [Improving Feature Attribution through Input-specific Network Pruning](https://mlsanity.com/paper/1911.11081/) (2019)
- [Octuplet Loss: Make Face Recognition Robust to Image Resolution](https://mlsanity.com/paper/2207.06726/) (2022)
- [Explainable vision transformer enabled convolutional neural network for plant disease identification: PlantXViT](https://mlsanity.com/paper/2207.07919/) (2022)

## Built on by (most cited first, 13 of 13)

- [Efficient Explainable Face Verification based on Similarity Score Argument Backpropagation](https://mlsanity.com/paper/2304.13409/) (2023)
- [Towards Visual Saliency Explanations of Face Verification](https://mlsanity.com/paper/2305.08546/) (2023)
- [Explainable Face Verification via Feature-Guided Gradient Backpropagation](https://mlsanity.com/paper/2403.04549/) (2024)
- [Discriminative Deep Feature Visualization for Explainable Face Recognition](https://mlsanity.com/paper/2306.00402/) (2023)
- [Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain](https://mlsanity.com/paper/2407.11941/) (2024)
- [Bridging Human Concepts and Computer Vision for Explainable Face Verification](https://mlsanity.com/paper/2403.08789/) (2024)
- [Non-Adaptive Adversarial Face Generation](https://mlsanity.com/paper/2507.12107/) (2025)
- [MLLM-based Textual Explanations for Face Comparison](https://mlsanity.com/paper/2603.16629/) (2026)
- [Tackling Face Verification Edge Cases: In-Depth Analysis and Human-Machine Fusion Approach](https://mlsanity.com/paper/2304.08134/) (2023)
- [Aligning Characteristic Descriptors with Images for Human-Expert-like Explainability](https://mlsanity.com/paper/2411.04008/) (2024)
- [Restricted Receptive Fields for Face Verification](https://mlsanity.com/paper/2510.10753/) (2025)
- [Casting the Net! Revisiting MasterFace Impersonation Attacks](https://mlsanity.com/paper/2608.06952/) (2026)
- [EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment](https://mlsanity.com/paper/2608.21486/) (2026)

## Abstract (arXiv)

Recently, face recognition systems have demonstrated remarkable performances and thus gained a vital role in our daily life. They already surpass human face verification accountability in many scenarios. However, they lack explanations for their predictions. Compared to human operators, typical face recognition network system generate only binary decisions without further explanation and insights into those decisions. This work focuses on explanations for face recognition systems, vital for developers and operators. First, we introduce a confidence score for those systems based on facial feature distances between two input images and the distribution of distances across a dataset. Secondly, we establish a novel visualization approach to obtain more meaningful predictions from a face recognition system, which maps the distance deviation based on a systematic occlusion of images. The result is blended with the original images and highlights similar and dissimilar facial regions. Lastly, we calculate confidence scores and explanation maps for several state-of-the-art face verification datasets and release the results on a web platform. We optimize the platform for a user-friendly interaction and hope to further improve the understanding of machine learning decisions. The source code is available on GitHub, and the web platform is publicly available at http://explainable-face-verification.ey.r.appspot.com.