# Explanatory Paradigms in Neural Networks

arXiv: https://arxiv.org/abs/2202.11838 · HTML: https://mlsanity.com/paper/2202.11838/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives](https://mlsanity.com/paper/1802.07623/) (2018)
- [COVID-CT-Dataset: A CT Scan Dataset about COVID-19](https://mlsanity.com/paper/2003.13865/) (2020)
- [Discovering Causal Signals in Images](https://mlsanity.com/paper/1605.08179/) (2016)
- [Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning](https://mlsanity.com/paper/1708.09843/) (2017)
- [Visual Causal Feature Learning](https://mlsanity.com/paper/1412.2309/) (2014)
- [Gradients as a Measure of Uncertainty in Neural Networks](https://mlsanity.com/paper/2008.08030/) (2020)
- [Backpropagated Gradient Representations for Anomaly Detection](https://mlsanity.com/paper/2007.09507/) (2020)
- [Interpreting CNN Knowledge via an Explanatory Graph](https://mlsanity.com/paper/1708.01785/) (2017)
- [CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognition](https://mlsanity.com/paper/1712.02463/) (2017)
- [Contrastive Explanations in Neural Networks](https://mlsanity.com/paper/2008.00178/) (2020)
- [Novelty Detection Through Model-Based Characterization of Neural Networks](https://mlsanity.com/paper/2008.06094/) (2020)
- [Distorted Representation Space Characterization Through Backpropagated Gradients](https://mlsanity.com/paper/1908.09998/) (2019)
- [Extracting Causal Visual Features for Limited label Classification](https://mlsanity.com/paper/2103.12322/) (2021)
- [Contrastive Reasoning in Neural Networks](https://mlsanity.com/paper/2103.12329/) (2021)

## Built on by (most cited first, 11 of 11)

- [Introspective Learning : A Two-Stage Approach for Inference in Neural Networks](https://mlsanity.com/paper/2209.08425/) (2022)
- [Probing the Purview of Neural Networks via Gradient Analysis](https://mlsanity.com/paper/2304.02834/) (2023)
- [TrajPRed: Trajectory Prediction with Region-based Relation Learning](https://mlsanity.com/paper/2404.06971/) (2024)
- [Explaining models relating objects and privacy](https://mlsanity.com/paper/2405.01646/) (2024)
- [VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability](https://mlsanity.com/paper/2406.00573/) (2024)
- [Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networks](https://mlsanity.com/paper/2302.05776/) (2023)
- [Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networks](https://mlsanity.com/paper/2405.13758/) (2024)
- [Are Objective Explanatory Evaluation metrics Trustworthy? An Adversarial Analysis](https://mlsanity.com/paper/2406.07820/) (2024)
- [CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface faults](https://mlsanity.com/paper/2408.11185/) (2024)
- [A unified framework for evaluating the robustness of machine-learning interpretability for prospect risking](https://mlsanity.com/paper/2602.14430/) (2026)
- [Explaining Representation Learning with Perceptual Components](https://mlsanity.com/paper/2406.06930/) (2024)

## Abstract (arXiv)

In this article, we present a leap-forward expansion to the study of explainability in neural networks by considering explanations as answers to abstract reasoning-based questions. With P as the prediction from a neural network, these questions are `Why P?', `What if not P?', and `Why P, rather than Q?' for a given contrast prediction Q. The answers to these questions are observed correlations, observed counterfactuals, and observed contrastive explanations respectively. Together, these explanations constitute the abductive reasoning scheme. We term the three explanatory schemes as observed explanatory paradigms. The term observed refers to the specific case of post-hoc explainability, when an explanatory technique explains the decision P after a trained neural network has made the decision P. The primary advantage of viewing explanations through the lens of abductive reasoning-based questions is that explanations can be used as reasons while making decisions. The post-hoc field of explainability, that previously only justified decisions, becomes active by being involved in the decision making process and providing limited, but relevant and contextual interventions. The contributions of this article are: (i) realizing explanations as reasoning paradigms, (ii) providing a probabilistic definition of observed explanations and their completeness, (iii) creating a taxonomy for evaluation of explanations, and (iv) positioning gradient-based complete explanainability's replicability and reproducibility across multiple applications and data modalities, (v) code repositories, publicly available at https://github.com/olivesgatech/Explanatory-Paradigms.