# Efficient Explainable Face Verification based on Similarity Score Argument Backpropagation

arXiv: https://arxiv.org/abs/2304.13409 · HTML: https://mlsanity.com/paper/2304.13409/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [CurricularFace: Adaptive Curriculum Learning Loss for Deep Face Recognition](https://mlsanity.com/paper/2004.00288/) (2020)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Partial FC: Training 10 Million Identities on a Single Machine](https://mlsanity.com/paper/2010.05222/) (2020)
- [ElasticFace: Elastic Margin Loss for Deep Face Recognition](https://mlsanity.com/paper/2109.09416/) (2021)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [A Comprehensive Study on Face Recognition Biases Beyond Demographics](https://mlsanity.com/paper/2103.01592/) (2021)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [A survey of face recognition techniques under occlusion](https://mlsanity.com/paper/2006.11366/) (2020)
- [PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation](https://mlsanity.com/paper/2108.10710/) (2021)
- [Surpassing Human-Level Face Verification Performance on LFW with GaussianFace](https://mlsanity.com/paper/1404.3840/) (2014)
- [xCos: An Explainable Cosine Metric for Face Verification Task](https://mlsanity.com/paper/2003.05383/) (2020)
- [Pixel-Level Face Image Quality Assessment for Explainable Face Recognition](https://mlsanity.com/paper/2110.11001/) (2021)
- [Explainable Model-Agnostic Similarity and Confidence in Face Verification](https://mlsanity.com/paper/2211.13735/) (2022)
- [Stating Comparison Score Uncertainty and Verification Decision Confidence Towards Transparent Face Recognition](https://mlsanity.com/paper/2210.10354/) (2022)
- [Explainability of the Implications of Supervised and Unsupervised Face Image Quality Estimations Through Activation Map Variation Analyses in Face Recognition Models](https://mlsanity.com/paper/2112.04827/) (2021)
- [Towards Explaining Demographic Bias through the Eyes of Face Recognition Models](https://mlsanity.com/paper/2208.13400/) (2022)

## Built on by (most cited first, 12 of 12)

- [EFaR 2023: Efficient Face Recognition Competition](https://mlsanity.com/paper/2308.04168/) (2023)
- [Explainable Face Verification via Feature-Guided Gradient Backpropagation](https://mlsanity.com/paper/2403.04549/) (2024)
- [From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing](https://mlsanity.com/paper/2409.16089/) (2024)
- [Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain](https://mlsanity.com/paper/2407.11941/) (2024)
- [Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition](https://mlsanity.com/paper/2408.10175/) (2024)
- [Towards Privacy-aware Mental Health AI Models: Advances, Challenges, and Opportunities](https://mlsanity.com/paper/2502.00451/) (2025)
- [Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systems](https://mlsanity.com/paper/2407.05983/) (2024)
- [Aligning Characteristic Descriptors with Images for Human-Expert-like Explainability](https://mlsanity.com/paper/2411.04008/) (2024)
- [A Responsible Face Recognition Approach for Small and Mid-Scale Systems Through Personalized Neural Networks](https://mlsanity.com/paper/2505.19920/) (2025)
- [Restricted Receptive Fields for Face Verification](https://mlsanity.com/paper/2510.10753/) (2025)
- [FAME: Feature Activation Map Explanation on Image Classification and Face Recognition](https://mlsanity.com/paper/2605.12017/) (2026)
- [EXPL-FR: Explaining Face Recognition Models via Vision-Language Alignment](https://mlsanity.com/paper/2608.21486/) (2026)

## Abstract (arXiv)

Explainable Face Recognition is gaining growing attention as the use of the technology is gaining ground in security-critical applications. Understanding why two faces images are matched or not matched by a given face recognition system is important to operators, users, anddevelopers to increase trust, accountability, develop better systems, and highlight unfair behavior. In this work, we propose xSSAB, an approach to back-propagate similarity score-based arguments that support or oppose the face matching decision to visualize spatial maps that indicate similar and dissimilar areas as interpreted by the underlying FR model. Furthermore, we present Patch-LFW, a new explainable face verification benchmark that enables along with a novel evaluation protocol, the first quantitative evaluation of the validity of similarity and dissimilarity maps in explainable face recognition approaches. We compare our efficient approach to state-of-the-art approaches demonstrating a superior trade-off between efficiency and performance. The code as well as the proposed Patch-LFW is publicly available at: https://github.com/marcohuber/xSSAB.