mlsanityData to 14 Oct 2026

Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?

arXiv 2105.10719 ↗First posted 22 May 2021cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample. People usually mask an input variable using its baseline value. However, there is no theory to examine whether baseline value faithfully represents the absence of an input variable, i.e., removing all signals from the input variable. Fortunately, recent studies show that the inference score of a DNN can be strictly disentangled into a set of causal patterns (or concepts) encoded by the DNN. Therefore, we propose to use causal patterns to examine the faithfulness of baseline values. More crucially, it is proven that causal patterns can be explained as the elementary rationale of the Shapley value. Furthermore, we propose a method to learn optimal baseline values, and experimental results have demonstrated its effectiveness.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 34 of its 76 references that are in the map. First posted on arXiv 22 May 2021.

Builds on

34 of 76 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  3. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  7. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  9. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 612
  10. 2019The many Shapley values for model explanationcited by 344
  11. 2015Understanding Neural Networks Through Deep Visualizationcited by 334
  12. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
22 more
  1. 2019Understanding Deep Networks via Extremal Perturbations and Smooth Maskscited by 259
  2. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  3. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 208
  4. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 200
  5. 2014Object Detectors Emerge in Deep Scene CNNscited by 179
  6. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  7. 2018Local Rule-Based Explanations of Black Box Decision Systemscited by 166
  8. 2017CausalGAN: Learning Causal Implicit Generative Models with Adversarial Trainingcited by 152
  9. 2019The Shapley Taylor Interaction Indexcited by 143
  10. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  11. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
  12. 2020Shapley explainability on the data manifoldcited by 88
  13. 2017Detecting Statistical Interactions from Neural Network Weightscited by 80
  14. 2019Neural Network Attributions: A Causal Perspectivecited by 77
  15. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  16. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  17. 2018Hierarchical interpretations for neural network predictionscited by 66
  18. 2017A causal framework for explaining the predictions of black-box sequence-to-sequence modelscited by 58
  19. 2020Interpreting and Boosting Dropout from a Game-Theoretic Viewcited by 34
  20. 2019Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Modelscited by 26
  21. 2018Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activationscited by 23
  22. 2020Feature Removal Is a Unifying Principle for Model Explanation Methodscited by 21

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 3; 2024: 4; 2025: 2.

10 papers.

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  2. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  3. 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
  4. 2023Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?cited by 11
  5. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
  6. 2023Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Conceptscited by 9
  7. 2025MaskSDM with Shapley values to improve flexibility, robustness, and explainability in species distribution modelingcited by 3
  8. 2023Towards Attributions of Input Variables in a Coalitioncited by 1
  9. 2024Evaluating the Correctness of Inference Patterns Used by LLMs for Judgmentcited by 1
  10. 2024Disentangling Regional Primitives for Image Generationcited by 0