A Dual-Perspective Approach to Evaluating Feature Attribution Methods
Not on the map. Why?
Abstract (arXiv)
Feature attribution methods attempt to explain neural network predictions by identifying relevant features. However, establishing a cohesive framework for assessing feature attribution remains a challenge. There are several views through which we can evaluate attributions. One principal lens is to observe the effect of perturbing attributed features on the model's behavior (i.e., faithfulness). While providing useful insights, existing faithfulness evaluations suffer from shortcomings that we reveal in this paper. In this work, we propose two new perspectives within the faithfulness paradigm that reveal intuitive properties: soundness and completeness. Soundness assesses the degree to which attributed features are truly predictive features, while completeness examines how well the resulting attribution reveals all the predictive features. The two perspectives are based on a firm mathematical foundation and provide quantitative metrics that are computable through efficient algorithms. We apply these metrics to mainstream attribution methods, offering a novel lens through which to analyze and compare feature attribution methods.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 37 of its 44 references that are in the map. First posted on arXiv 17 Aug 2023.
Builds on
37 of 44 references.
References · 37
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
- 2017Axiomatic Attribution for Deep Networkscited by 3,384
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
- 2018Sanity Checks for Saliency Mapscited by 1,156
- 2017SmoothGrad: removing noise by adding noisecited by 1,109
25 more
- 2014Striving for Simplicity: The All Convolutional Netcited by 990
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
- 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
- 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
- 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 476
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
- 2019The many Shapley values for model explanationcited by 344
- 2016Top-down Neural Attention by Excitation Backpropcited by 321
- 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
- 2022The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspectivecited by 159
- 2020Drug discovery with explainable artificial intelligencecited by 118
- 2020Restricting the Flow: Information Bottlenecks for Attributioncited by 110
- 2022A Consistent and Efficient Evaluation Strategy for Attribution Methodscited by 102
- 2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
- 2021The effectiveness of feature attribution methods and its correlation with automatic evaluation scorescited by 88
- 2022A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributionscited by 64
- 2021Topology Preserving Local Road Network Estimation from Single Onboard Camera Imagecited by 34
- 2021Neural Response Interpretation through the Lens of Critical Pathwayscited by 32
- 2022Uncertainty-Aware Deep Multi-View Photometric Stereocited by 30
- 2022Do Explanations Explain? Model Knows Bestcited by 20
- 2022Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithmscited by 19
- 2021Fine-Grained Neural Network Explanation by Identifying Input Features with Predictive Informationcited by 17
- 2022A Psychological Theory of Explainabilitycited by 10
- 2021Explaining COVID-19 and Thoracic Pathology Model Predictions by Identifying Informative Input Featurescited by 9
- 2021Towards Semantic Interpretation of Thoracic Disease and COVID-19 Diagnosis Modelscited by 9
Built on by
Citing papers per year: 2026: 1.
1 paper.