# DANCE: Enhancing saliency maps using decoys

arXiv: https://arxiv.org/abs/2002.00526 · HTML: https://mlsanity.com/paper/2002.00526/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2020.

## Built on by (most cited first, 10 of 10)

- [Learning to Identify Critical States for Reinforcement Learning from Videos](https://mlsanity.com/paper/2308.07795/) (2023)
- [Improving Attribution Methods by Learning Submodular Functions](https://mlsanity.com/paper/2104.09073/) (2021)
- [On the Robustness of Explanations of Deep Neural Network Models: A Survey](https://mlsanity.com/paper/2211.04780/) (2022)
- [Path Choice Matters for Clear Attribution in Path Methods](https://mlsanity.com/paper/2401.10442/) (2024)
- [AdaptGrad: Adaptive Sampling to Reduce Noise](https://mlsanity.com/paper/2410.07711/) (2024)
- [Provable Robust Saliency-based Explanations](https://mlsanity.com/paper/2212.14106/) (2022)
- [DeepROCK: Error-controlled interaction detection in deep neural networks](https://mlsanity.com/paper/2309.15319/) (2023)
- [Robust Ranking Explanations](https://mlsanity.com/paper/2307.04024/) (2023)
- [Axiomatization of Gradient Smoothing in Neural Networks](https://mlsanity.com/paper/2407.00371/) (2024)
- [Error-controlled non-additive interaction discovery in machine learning models](https://mlsanity.com/paper/2408.17016/) (2024)

## Abstract (arXiv)

Saliency methods can make deep neural network predictions more interpretable by identifying a set of critical features in an input sample, such as pixels that contribute most strongly to a prediction made by an image classifier. Unfortunately, recent evidence suggests that many saliency methods poorly perform, especially in situations where gradients are saturated, inputs contain adversarial perturbations, or predictions rely upon inter-feature dependence. To address these issues, we propose a framework that improves the robustness of saliency methods by following a two-step procedure. First, we introduce a perturbation mechanism that subtly varies the input sample without changing its intermediate representations. Using this approach, we can gather a corpus of perturbed data samples while ensuring that the perturbed and original input samples follow the same distribution. Second, we compute saliency maps for the perturbed samples and propose a new method to aggregate saliency maps. With this design, we offset the gradient saturation influence upon interpretation. From a theoretical perspective, we show the aggregated saliency map could not only capture inter-feature dependence but, more importantly, robustify interpretation against previously described adversarial perturbation methods. Following our theoretical analysis, we present experimental results suggesting that, both qualitatively and quantitatively, our saliency method outperforms existing methods.