# Contrastive Explanations in Neural Networks

arXiv: https://arxiv.org/abs/2008.00178 · HTML: https://mlsanity.com/paper/2008.00178/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2020.

## Built on by (most cited first, 26 of 26)

- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [Towards Relatable Explainable AI with the Perceptual Process](https://mlsanity.com/paper/2112.14005/) (2021)
- [OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics](https://mlsanity.com/paper/2209.11195/) (2022)
- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Attribution-based XAI Methods in Computer Vision: A Review](https://mlsanity.com/paper/2211.14736/) (2022)
- [Gradient-Based Adversarial and Out-of-Distribution Detection](https://mlsanity.com/paper/2206.08255/) (2022)
- [Extracting Causal Visual Features for Limited label Classification](https://mlsanity.com/paper/2103.12322/) (2021)
- [Contrastive Reasoning in Neural Networks](https://mlsanity.com/paper/2103.12329/) (2021)
- [Explanatory Paradigms in Neural Networks](https://mlsanity.com/paper/2202.11838/) (2022)
- [Gaussian Switch Sampling: A Second Order Approach to Active Learning](https://mlsanity.com/paper/2302.12018/) (2023)
- [Open-Set Recognition with Gradient-Based Representations](https://mlsanity.com/paper/2206.08229/) (2022)
- [Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism](https://mlsanity.com/paper/2209.11189/) (2022)
- [Explaining Deep Models through Forgettable Learning Dynamics](https://mlsanity.com/paper/2301.04221/) (2023)
- [HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms](https://mlsanity.com/paper/2410.23200/) (2024)
- [Hierarchical and Multimodal Data for Daily Activity Understanding](https://mlsanity.com/paper/2504.17696/) (2025)
- [Example Forgetting: A Novel Approach to Explain and Interpret Deep Neural Networks in Seismic Interpretation](https://mlsanity.com/paper/2302.14644/) (2023)
- [Two-Stage Holistic and Contrastive Explanation of Image Classification](https://mlsanity.com/paper/2306.06339/) (2023)
- [VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability](https://mlsanity.com/paper/2406.00573/) (2024)
- [Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networks](https://mlsanity.com/paper/2302.05776/) (2023)
- [Are Objective Explanatory Evaluation metrics Trustworthy? An Adversarial Analysis](https://mlsanity.com/paper/2406.07820/) (2024)
- [CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface faults](https://mlsanity.com/paper/2408.11185/) (2024)
- [Towards Fine-Grained Interpretability: Counterfactual Explanations for Misclassification with Saliency Partition](https://mlsanity.com/paper/2511.07974/) (2025)
- [Explainable Machine Learning for Hydrocarbon Prospect Risking](https://mlsanity.com/paper/2212.07563/) (2022)
- [VAE-CE: Visual Contrastive Explanation using Disentangled VAEs](https://mlsanity.com/paper/2108.09159/) (2021)
- [Toward Simple and Robust Contrastive Explanations for Image Classification by Leveraging Instance Similarity and Concept Relevance](https://mlsanity.com/paper/2506.23975/) (2025)
- [Gradient based Severity Labeling for Biomarker Classification in OCT](https://mlsanity.com/paper/2602.19907/) (2026)

## Abstract (arXiv)

Visual explanations are logical arguments based on visual features that justify the predictions made by neural networks. Current modes of visual explanations answer questions of the form `Why   P?'. These Why questions operate under broad contexts thereby providing answers that are irrelevant in some cases. We propose to constrain these Why questions based on some context Q so that our explanations answer contrastive questions of the form `Why   P,  rather   than   Q?'. In this paper, we formalize the structure of contrastive visual explanations for neural networks. We define contrast based on neural networks and propose a methodology to extract defined contrasts. We then use the extracted contrasts as a plug-in on top of existing `Why   P?' techniques, specifically Grad-CAM. We demonstrate their value in analyzing both networks and data in applications of large-scale recognition, fine-grained recognition, subsurface seismic analysis, and image quality assessment.