AttributionLab: Faithfulness of Feature Attribution Under Controllable Environments
Not on the map. Why?
Abstract (arXiv)
Feature attribution explains neural network outputs by identifying relevant input features. The attribution has to be faithful, meaning that the attributed features must mirror the input features that influence the output. One recent trend to test faithfulness is to fit a model on designed data with known relevant features and then compare attributions with ground truth input features.This idea assumes that the model learns to use all and only these designed features, for which there is no guarantee. In this paper, we solve this issue by designing the network and manually setting its weights, along with designing data. The setup, AttributionLab, serves as a sanity check for faithfulness: If an attribution method is not faithful in a controlled environment, it can be unreliable in the wild. The environment is also a laboratory for controlled experiments by which we can analyze attribution methods and suggest improvements.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 10 Oct 2023.
Builds on
42 of 56 references.
References · 42
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
30 more
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2013Visualizing and Understanding Convolutional Networkscited by 2,290
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
- 2014Striving for Simplicity: The All Convolutional Netcited by 990
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
- 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
- 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
- 2013Do Deep Nets Really Need to be Deep?cited by 474
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
- 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
- 2019The many Shapley values for model explanationcited by 344
- 2016Top-down Neural Attention by Excitation Backpropcited by 321
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
- 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
- 2020Ground Truth Evaluation of Neural Network Explanations with CLEVR-XAIcited by 126
- 2020Restricting the Flow: Information Bottlenecks for Attributioncited by 110
- 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 102
- 2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
- 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
- 2022Evaluating Explainability for Graph Neural Networkscited by 87
- 2019Robust Attribution Regularizationcited by 47
- 2021Neural Response Interpretation through the Lens of Critical Pathwayscited by 32
- 2022Do Explanations Explain? Model Knows Bestcited by 20
- 2021Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Informationcited by 17
- 2021Two4Two: Evaluating Interpretable Machine Learning - A Synthetic Dataset For Controlled Experimentscited by 1
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.