# Towards Visual Saliency Explanations of Face Verification

arXiv: https://arxiv.org/abs/2305.08546 · HTML: https://mlsanity.com/paper/2305.08546/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 15 May 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [AdaFace: Quality Adaptive Margin for Face Recognition](https://mlsanity.com/paper/2204.00964/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [Top-down Neural Attention by Excitation Backprop](https://mlsanity.com/paper/1608.00507/) (2016)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [Tell Me Where to Look: Guided Attention Inference Network](https://mlsanity.com/paper/1802.10171/) (2018)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [Black-box Explanation of Object Detectors via Saliency Maps](https://mlsanity.com/paper/2006.03204/) (2020)
- [The Promise and Peril of Human Evaluation for Model Interpretability](https://mlsanity.com/paper/1711.07414/) (2017)
- [Explainable Face Recognition](https://mlsanity.com/paper/2008.00916/) (2020)
- [Towards Interpretable Face Recognition](https://mlsanity.com/paper/1805.00611/) (2018)
- [Visualizing Deep Similarity Networks](https://mlsanity.com/paper/1901.00536/) (2019)
- [xCos: An Explainable Cosine Metric for Face Verification Task](https://mlsanity.com/paper/2003.05383/) (2020)
- [Explainable Model-Agnostic Similarity and Confidence in Face Verification](https://mlsanity.com/paper/2211.13735/) (2022)
- [When Face Recognition Meets Occlusion: A New Benchmark](https://mlsanity.com/paper/2103.02805/) (2021)

## Built on by (most cited first, 11 of 11)

- [Explainable Face Verification via Feature-Guided Gradient Backpropagation](https://mlsanity.com/paper/2403.04549/) (2024)
- [From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing](https://mlsanity.com/paper/2409.16089/) (2024)
- [Discriminative Deep Feature Visualization for Explainable Face Recognition](https://mlsanity.com/paper/2306.00402/) (2023)
- [Beyond Spatial Explanations: Explainable Face Recognition in the Frequency Domain](https://mlsanity.com/paper/2407.11941/) (2024)
- [MLLM-based Textual Explanations for Face Comparison](https://mlsanity.com/paper/2603.16629/) (2026)
- [Towards A Comprehensive Visual Saliency Explanation Framework for AI-based Face Recognition Systems](https://mlsanity.com/paper/2407.05983/) (2024)
- [Explainable embeddings with Distance Explainer](https://mlsanity.com/paper/2505.15516/) (2025)
- [Restricted Receptive Fields for Face Verification](https://mlsanity.com/paper/2510.10753/) (2025)
- [Adversarial Camouflage](https://mlsanity.com/paper/2603.21867/) (2026)
- [FAME: Feature Activation Map Explanation on Image Classification and Face Recognition](https://mlsanity.com/paper/2605.12017/) (2026)
- [Vision Language Model Fusion for Explainable Face Recognition](https://mlsanity.com/paper/2608.24430/) (2026)

## Abstract (arXiv)

In the past years, deep convolutional neural networks have been pushing the frontier of face recognition (FR) techniques in both verification and identification scenarios. Despite the high accuracy, they are often criticized for lacking explainability. There has been an increasing demand for understanding the decision-making process of deep face recognition systems. Recent studies have investigated the usage of visual saliency maps as an explanation, but they often lack a discussion and analysis in the context of face recognition. This paper concentrates on explainable face verification tasks and conceives a new explanation framework. Firstly, a definition of the saliency-based explanation method is provided, which focuses on the decisions made by the deep FR model. Secondly, a new model-agnostic explanation method named CorrRISE is proposed to produce saliency maps, which reveal both the similar and dissimilar regions of any given pair of face images. Then, an evaluation methodology is designed to measure the performance of general visual saliency explanation methods in face verification. Finally, substantial visual and quantitative results have shown that the proposed CorrRISE method demonstrates promising results in comparison with other state-of-the-art explainable face verification approaches.