Suboptimal Shapley Value Explanations

Deep Neural Networks (DNNs) have demonstrated strong capacity in supporting a wide variety of applications. Shapley value has emerged as a prominent tool to analyze feature importance to help people understand the inference process of deep neural models. Computing Shapley value function requires choosing a baseline to represent feature's missingness. However, existing random and conditional baselines could negatively influence the explanation. In this paper, by analyzing the suboptimality of different baselines, we identify the problematic baseline where the asymmetric interaction between bmx'i (the replacement of the faithful influential feature) and other features has significant directional bias toward the model's output, and conclude that p(y|bmx'i) = p(y) potentially minimizes the asymmetric interaction involving bmx'i. We further generalize the uninformativeness of bmx'i toward the label space L to avoid estimating p(y) and design a simple uncertainty-based reweighting mechanism to accelerate the computation process. We conduct experiments on various NLP tasks and our quantitative analysis demonstrates the effectiveness of the proposed uncertainty-based reweighting mechanism. Furthermore, by measuring the consistency of explanations generated by explainable methods and human, we highlight the disparity between model inference and human understanding.