# Explaining Models by Propagating Shapley Values of Local Components

arXiv: https://arxiv.org/abs/1911.11888 · HTML: https://mlsanity.com/paper/1911.11888/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 29 of 29)

- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [HINT: Hierarchical Neuron Concept Explainer](https://mlsanity.com/paper/2203.14196/) (2022)
- [Fast Hierarchical Games for Image Explanations](https://mlsanity.com/paper/2104.06164/) (2021)
- [Towards Zero-shot Sign Language Recognition](https://mlsanity.com/paper/2201.05914/) (2022)
- [Explainable Artificial Intelligence (XAI) for Internet of Things: A Survey](https://mlsanity.com/paper/2206.04800/) (2022)
- [A k-additive Choquet integral-based approach to approximate the SHAP values for local interpretability in machine learning](https://mlsanity.com/paper/2211.02166/) (2022)
- [SHAP Meets Tensor Networks: Provably Tractable Explanations with Parallelism](https://mlsanity.com/paper/2510.21599/) (2025)
- [Uncertainty aware and explainable diagnosis of retinal disease](https://mlsanity.com/paper/2101.12041/) (2021)
- [Recent Advances in Malware Detection: Graph Learning and Explainability](https://mlsanity.com/paper/2502.10556/) (2025)
- [Toward Understanding the Disagreement Problem in Neural Network Feature Attribution](https://mlsanity.com/paper/2404.11330/) (2024)
- [Accurate Explanation Model for Image Classifiers using Class Association Embedding](https://mlsanity.com/paper/2406.07961/) (2024)
- [Psychophysiological Arousal in Young Children Who Stutter: An Interpretable AI Approach](https://mlsanity.com/paper/2208.08859/) (2022)
- [Comparing Baseline Shapley and Integrated Gradients for Local Explanation: Some Additional Insights](https://mlsanity.com/paper/2208.06096/) (2022)
- [On marginal feature attributions of tree-based models](https://mlsanity.com/paper/2302.08434/) (2023)
- [A New Baseline Assumption of Integated Gradients Based on Shaply value](https://mlsanity.com/paper/2310.04821/) (2023)
- [Satellite Image and Machine Learning based Knowledge Extraction in the Poverty and Welfare Domain](https://mlsanity.com/paper/2203.01068/) (2022)
- [T-Explainer: A Model-Agnostic Explainability Framework Based on Gradients](https://mlsanity.com/paper/2404.16495/) (2024)
- [Applications of Explainable artificial intelligence in Earth system science](https://mlsanity.com/paper/2406.11882/) (2024)
- [A Review of Global Sensitivity Analysis Methods and a comparative case study on Digit Classification](https://mlsanity.com/paper/2406.16975/) (2024)
- [Guidelines For The Choice Of The Baseline in XAI Attribution Methods](https://mlsanity.com/paper/2503.19813/) (2025)
- [Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2507.07148/) (2025)
- [A Comparative Approach to Explainable Artificial Intelligence Methods in Application to High-Dimensional Electronic Health Records: Examining the Usability of XAI](https://mlsanity.com/paper/2103.04951/) (2021)
- [Novel Multicolumn Kernel Extreme Learning Machine for Food Detection via Optimal Features from CNN](https://mlsanity.com/paper/2205.07348/) (2022)
- [Developing A Visual-Interactive Interface for Electronic Health Record Labeling: An Explainable Machine Learning Approach](https://mlsanity.com/paper/2209.12778/) (2022)
- [Explainable AI for Autism Diagnosis: Identifying Critical Brain Regions Using fMRI Data](https://mlsanity.com/paper/2409.15374/) (2024)
- [Understanding molecular ratios in the carbon and oxygen poor outer Milky Way with interpretable machine learning](https://mlsanity.com/paper/2505.08410/) (2025)
- [Verified SHAP: Provable Bounds for Exact Shapley Values of Neural Networks](https://mlsanity.com/paper/2605.24084/) (2026)

## Abstract (arXiv)

In healthcare, making the best possible predictions with complex models (e.g., neural networks, ensembles/stacks of different models) can impact patient welfare. In order to make these complex models explainable, we present DeepSHAP for mixed model types, a framework for layer wise propagation of Shapley values that builds upon DeepLIFT (an existing approach for explaining neural networks). We show that in addition to being able to explain neural networks, this new framework naturally enables attributions for stacks of mixed models (e.g., neural network feature extractor into a tree model) as well as attributions of the loss. Finally, we theoretically justify a method for obtaining attributions with respect to a background distribution (under a Shapley value framework).