# Explain To Me: Salience-Based Explainability for Synthetic Face Detection Models

arXiv: https://arxiv.org/abs/2303.11969 · HTML: https://mlsanity.com/paper/2303.11969/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [The What-If Tool: Interactive Probing of Machine Learning Models](https://mlsanity.com/paper/1907.04135/) (2019)
- [Axiom-based Grad-CAM: Towards Accurate Visualization and Explanation of CNNs](https://mlsanity.com/paper/2008.02312/) (2020)
- [Smooth Grad-CAM++: An Enhanced Inference Level Visualization Technique for Deep Convolutional Neural Network Models](https://mlsanity.com/paper/1908.01224/) (2019)
- [DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling](https://mlsanity.com/paper/2105.12441/) (2021)
- [SS-CAM: Smoothed Score-CAM for Sharper Visual Feature Localization](https://mlsanity.com/paper/2006.14255/) (2020)
- [CYBORG: Blending Human Saliency Into the Loss Improves Deep Learning](https://mlsanity.com/paper/2112.00686/) (2021)
- [IS-CAM: Integrated Score-CAM for axiomatic-based explanations](https://mlsanity.com/paper/2010.03023/) (2020)
- [Human-Aided Saliency Maps Improve Generalization of Deep Learning](https://mlsanity.com/paper/2105.03492/) (2021)
- [A Methodology for Creating AI FactSheets](https://mlsanity.com/paper/2006.13796/) (2020)

## Built on by (most cited first, 8 of 8)

- [Second FRCSyn-onGoing: Winning Solutions and Post-Challenge Analysis to Improve Face Recognition with Synthetic Data](https://mlsanity.com/paper/2412.01383/) (2024)
- [MENTOR: Human Perception-Guided Pretraining for Increased Generalization](https://mlsanity.com/paper/2310.19545/) (2023)
- [Grains of Saliency: Optimizing Saliency-based Training of Biometric Attack Detection Models](https://mlsanity.com/paper/2405.00650/) (2024)
- [Taking Training Seriously: Human Guidance and Management-Based Regulation of Artificial Intelligence](https://mlsanity.com/paper/2402.08466/) (2024)
- [Saliency-Guided Training for Fingerprint Presentation Attack Detection](https://mlsanity.com/paper/2505.02176/) (2025)
- [AutoSIGHT: Automatic Eye Tracking-based System for Immediate Grading of Human experTise](https://mlsanity.com/paper/2508.01015/) (2025)
- [Almost Right: Making First-Layer Kernels Nearly Orthogonal Improves Model Generalization](https://mlsanity.com/paper/2504.16362/) (2025)
- [SAGE: Saliency-Guided Contrastive Embeddings](https://mlsanity.com/paper/2511.12744/) (2025)

## Abstract (arXiv)

The performance of convolutional neural networks has continued to improve over the last decade. At the same time, as model complexity grows, it becomes increasingly more difficult to explain model decisions. Such explanations may be of critical importance for reliable operation of human-machine pairing setups, or for model selection when the "best" model among many equally-accurate models must be established. Saliency maps represent one popular way of explaining model decisions by highlighting image regions models deem important when making a prediction. However, examining salience maps at scale is not practical. In this paper, we propose five novel methods of leveraging model salience to explain a model behavior at scale. These methods ask: (a) what is the average entropy for a model's salience maps, (b) how does model salience change when fed out-of-set samples, (c) how closely does model salience follow geometrical transformations, (d) what is the stability of model salience across independent training runs, and (e) how does model salience react to salience-guided image degradations. To assess the proposed measures on a concrete and topical problem, we conducted a series of experiments for the task of synthetic face detection with two types of models: those trained traditionally with cross-entropy loss, and those guided by human salience when training to increase model generalizability. These two types of models are characterized by different, interpretable properties of their salience maps, which allows for the evaluation of the correctness of the proposed measures. We offer source codes for each measure along with this paper.