Corrupting Neuron Explanations of Deep Visual Features
Not on the map. Why?
Abstract (arXiv)
The inability of DNNs to explain their black-box behavior has led to a recent surge of explainability methods. However, there are growing concerns that these explainability methods are not robust and trustworthy. In this work, we perform the first robustness analysis of Neuron Explanation Methods under a unified pipeline and show that these explanations can be significantly corrupted by random noises and well-designed perturbations added to their probing data. We find that even adding small random noise with a standard deviation of 0.02 can already change the assigned concepts of up to 28% neurons in the deeper layers. Furthermore, we devise a novel corruption algorithm and show that our algorithm can manipulate the explanation of more than 80% neurons by poisoning less than 10% of probing data. This raises the concern of trusting Neuron Explanation Methods in real-life safety and fairness critical applications.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 25 Oct 2023.
Builds on
17 of 20 references.
References · 17
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2015Learning Deep Features for Discriminative Localizationcited by 2,430
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
- 2017SmoothGrad: removing noise by adding noisecited by 1,097
- 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
- 2017Interpretation of Neural Networks is Fragilecited by 443
- 2018On the Robustness of Interpretability Methodscited by 310
- 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
- 2020Compositional Explanations of Neuronscited by 172
5 more
- 2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 156
- 2022Natural Language Descriptions of Deep Visual Featurescited by 136
- 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
- 2021Robust Feature-Level Adversaries are Interpretability Toolscited by 27
- 2019Analyzing the Interpretability Robustness of Self-Explaining Modelscited by 5
Built on by
Citing papers per year: 2024: 2.
2 papers.