# Gradients of Counterfactuals

arXiv: https://arxiv.org/abs/1611.02639 · HTML: https://mlsanity.com/paper/1611.02639/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2016.

## Built on by (most cited first, 30 of 31)

- [Do Input Gradients Highlight Discriminative Features?](https://mlsanity.com/paper/2102.12781/) (2021)
- [Finding Discriminative Filters for Specific Degradations in Blind Super-Resolution](https://mlsanity.com/paper/2108.01070/) (2021)
- [Abs-CAM: A Gradient Optimization Interpretable Approach for Explanation of Convolutional Neural Networks](https://mlsanity.com/paper/2207.03648/) (2022)
- [Restore Anything with Masks: Leveraging Mask Image Modeling for Blind All-in-One Image Restoration](https://mlsanity.com/paper/2409.19403/) (2024)
- [Surrogate Model-Based Explainability Methods for Point Cloud NNs](https://mlsanity.com/paper/2107.13459/) (2021)
- [A Survey on Understanding, Visualizations, and Explanation of Deep Neural Networks](https://mlsanity.com/paper/2102.01792/) (2021)
- [Projective Ranking-based GNN Evasion Attacks](https://mlsanity.com/paper/2202.12993/) (2022)
- [Explainable Artificial Intelligence for Medical Applications: A Review](https://mlsanity.com/paper/2412.01829/) (2024)
- [Instance-wise or Class-wise? A Tale of Neighbor Shapley for Concept-based Explanation](https://mlsanity.com/paper/2109.01369/) (2021)
- [Explainability-Aware One Point Attack for Point Cloud Neural Networks](https://mlsanity.com/paper/2110.04158/) (2021)
- [Efficient Unified Demosaicing for Bayer and Non-Bayer Patterned Image Sensors](https://mlsanity.com/paper/2307.10667/) (2023)
- [A new interpretable unsupervised anomaly detection method based on residual explanation](https://mlsanity.com/paper/2103.07953/) (2021)
- [Expected Grad-CAM: Towards gradient faithfulness](https://mlsanity.com/paper/2406.01274/) (2024)
- [Estimation and Applications of Quantiles in Deep Binary Classification](https://mlsanity.com/paper/2102.06575/) (2021)
- [Maximum Entropy Baseline for Integrated Gradients](https://mlsanity.com/paper/2204.05948/) (2022)
- [A Tour of Visualization Techniques for Computer Vision Datasets](https://mlsanity.com/paper/2204.08601/) (2022)
- [Jacobian Norm with Selective Input Gradient Regularization for Improved and Interpretable Adversarial Defense](https://mlsanity.com/paper/2207.13036/) (2022)
- [Forward Learning for Gradient-based Black-box Saliency Map Generation](https://mlsanity.com/paper/2403.15603/) (2024)
- [Balancing Robustness and Sensitivity using Feature Contrastive Learning](https://mlsanity.com/paper/2105.09394/) (2021)
- [Clustering-Based Interpretation of Deep ReLU Network](https://mlsanity.com/paper/2110.06593/) (2021)
- [Brain Structural Saliency Over The Ages](https://mlsanity.com/paper/2202.11690/) (2022)
- [Understanding CNNs from excitations](https://mlsanity.com/paper/2205.00932/) (2022)
- [Generalizability Analysis of Graph-based Trajectory Predictor with Vectorized Representation](https://mlsanity.com/paper/2208.03578/) (2022)
- [Adversarial 3D Virtual Patches using Integrated Gradients](https://mlsanity.com/paper/2406.00282/) (2024)
- [Unlearning-based Neural Interpretations](https://mlsanity.com/paper/2410.08069/) (2024)
- [RAM++: Robust Representation Learning via Adaptive Mask for All-in-One Image Restoration](https://mlsanity.com/paper/2509.12039/) (2025)
- [Logic Constraints to Feature Importances](https://mlsanity.com/paper/2110.06596/) (2021)
- [The Generalizability of Explanations](https://mlsanity.com/paper/2302.11965/) (2023)
- [DeepCSHAP: Utilizing Shapley Values to Explain Deep Complex-Valued Neural Networks](https://mlsanity.com/paper/2403.08428/) (2024)
- [Explainability-Aware Frustum Attack: Exposing Structural Vulnerabilities in LiDAR-Based 3D Object Detectors](https://mlsanity.com/paper/2606.29963/) (2026)

## Abstract (arXiv)

Gradients have been used to quantify feature importance in machine learning models. Unfortunately, in nonlinear deep networks, not only individual neurons but also the whole network can saturate, and as a result an important input feature can have a tiny gradient. We study various networks, and observe that this phenomena is indeed widespread, across many inputs. We propose to examine interior gradients, which are gradients of counterfactual inputs constructed by scaling down the original input. We apply our method to the GoogleNet architecture for object recognition in images, as well as a ligand-based virtual screening network with categorical features and an LSTM based language model for the Penn Treebank dataset. We visualize how interior gradients better capture feature importance. Furthermore, interior gradients are applicable to a wide variety of deep networks, and have the attribution property that the feature importance scores sum to the the prediction score. Best of all, interior gradients can be computed just as easily as gradients. In contrast, previous methods are complex to implement, which hinders practical adoption.