mlsanityData to 7 Oct 2026

Explaining a Series of Models by Propagating Shapley Values

arXiv 2105.00108 ↗First posted 30 Apr 2021cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Local feature attribution methods are increasingly used to explain complex machine learning models. However, current methods are limited because they are extremely expensive to compute or are not capable of explaining a distributed series of models where each model is owned by a separate institution. The latter is particularly important because it often arises in finance where explanations are mandated. Here, we present DeepSHAP, a tractable method to propagate local feature attributions through complex series of models based on a connection to the Shapley value. We evaluate DeepSHAP across biological, health, and financial datasets to show that it provides equally salient explanations an order of magnitude faster than existing model-agnostic attribution techniques and demonstrate its use in an important distributed series of models setting.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 30 of its 91 references that are in the map. First posted on arXiv 30 Apr 2021.

Builds on

30 of 91 references.

References · 30

  1. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  2. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  4. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  6. 2018Sanity Checks for Saliency Mapscited by 1,149
  7. 2016The Mythos of Model Interpretabilitycited by 716
  8. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  9. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  10. 2019The many Shapley values for model explanationcited by 341
  11. 2019Interpretable machine learning: definitions, methods, and applicationscited by 303
  12. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
18 more
  1. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
  2. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 198
  3. 2019Feature relevance quantification in explainable AI: A causal problemcited by 164
  4. 2020Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regressioncited by 153
  5. 2020True to the Model or True to the Data?cited by 123
  6. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  7. 2020Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Modelscited by 107
  8. 2018L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Datacited by 100
  9. 2019CXPlain: Causal Explanations for Model Interpretation under Uncertaintycited by 98
  10. 2019When Explanations Lie: Why Many Modified BP Attributions Failcited by 92
  11. 2019Explainable AI for Trees: From Local Explanations to Global Understandingcited by 64
  12. 2020Efficient nonparametric statistical inference on population feature importance using Shapley valuescited by 61
  13. 2018The Challenge of Crafting Intelligible Intelligencecited by 46
  14. 2021Shapley Explanation Networkscited by 41
  15. 2019Explaining Models by Propagating Shapley Values of Local Componentscited by 29
  16. 2019Explaining black box decisions by Shapley cohort refinementcited by 26
  17. 2016Improved prediction accuracy for disease risk mapping using Gaussian Process stacked generalisationcited by 8
  18. 2019Generalized Integrated Gradients: A practical method for explaining diverse ensemblescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 15; 2025: 10; 2026: 1.

37 papers.

Sort:

By area:

  1. 2022Algorithms to estimate Shapley value feature attributionscited by 114
  2. 2023CausalTime: Realistically Generated Time-series for Benchmarking of Causal Discoverycited by 35
  3. 2023A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Themcited by 15
  4. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
  5. 2023Negative Flux Aggregation to Estimate Feature Attributionscited by 7
  6. 2023Sparse learned kernels for interpretable and efficient medical time series processingcited by 6
  7. 2023MRxaI: Black-Box Explainability for Image Classifiers in a Medical Settingcited by 6
  8. 2024Beyond Label Attention: Transparency in Language Models for Automated Medical Coding via Dictionary Learningcited by 5
  9. 2024SHAP zero Explains Biological Sequence Models with Near-zero Marginal Cost for Future Queriescited by 4
  10. 2022Model free variable importance for high dimensional datacited by 3
  11. 2023A New Baseline Assumption of Integated Gradients Based on Shaply valuecited by 3
  12. 2024DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Predictioncited by 3
25 more
  1. 2025Understanding Data Influence with Differential Approximationcited by 3
  2. 2024Reliable and scalable variable importance estimation via warm-start and early stoppingcited by 2
  3. 2024A Comprehensive Study of Shapley Value in Data Analyticscited by 2
  4. 2025Attribution Explanations for Deep Neural Networks: A Theoretical Perspectivecited by 2
  5. 2025Explaining Concept Drift through the Evolution of Group Counterfactualscited by 2
  6. 2023A Quantitatively Interpretable Model for Alzheimer's Disease Prediction Using Deep Counterfactualscited by 1
  7. 2023MixEHR-SurG: a joint proportional hazard and guided topic model for inferring mortality-associated topics from electronic health recordscited by 1
  8. 2024Bayesian Networks for Causal Analysis in Socioecological Systemscited by 1
  9. 2024Prospector Heads: Generalized Feature Attribution for Large Models & Datacited by 1
  10. 2024Energy-Based Model for Accurate Estimation of Shapley Values in Feature Attributioncited by 1
  11. 2024T-Explainer: A Model-Agnostic Explainability Framework Based on Gradientscited by 1
  12. 2024Feature importance to explain multimodal prediction models. A clinical use casecited by 1
  13. 2024Feature Importance in Pedestrian Intention Prediction: A Context-Aware Reviewcited by 1
  14. 2024Recent advances in interpretable machine learning using structure-based protein representationscited by 1
  15. 2025Towards Reliable and Interpretable Traffic Crash Pattern Prediction and Safety Interventions Using Customized Large Language Modelscited by 1
  16. 2025Holistic Explainable AI (H-XAI): Extending Transparency Beyond Developers in AI-Driven Decision Makingcited by 1
  17. 2023Evaluating the overall sensitivity of saliency-based explanation methodscited by 0
  18. 2024eipy: An Open-Source Python Package for Multi-modal Data Integration using Heterogeneous Ensemblescited by 0
  19. 2024Length-scale study in deep learning prediction for non-small cell lung cancer brain metastasiscited by 0
  20. 2024Development and Validation of a Deep-Learning Model for Differential Treatment Benefit Prediction for Adults with Major Depressive Disorder Deployed in the Artificial Intelligence in Depression Medication Enhancement (AIDME) Studycited by 0
  21. 2025Generalised Label-free Artefact Cleaning for Real-time Medical Pulsatile Time Seriescited by 0
  22. 2025Study of Robust Features in Formulating Guidance for Heuristic Algorithms for Solving the Vehicle Routing Problemcited by 0
  23. 2025Influence-Guided Concolic Testing of Transformer Robustnesscited by 0
  24. 2025UbiQVision: Quantifying Uncertainty in XAI for Image Recognitioncited by 0
  25. 2026Feature Attribution in Directed Acyclic Graphs Using Edge Interventioncited by 0