# Explaining black box decisions by Shapley cohort refinement

arXiv: https://arxiv.org/abs/1911.00467 · HTML: https://mlsanity.com/paper/1911.00467/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 26 of 26)

- [Algorithms to estimate Shapley value feature attributions](https://mlsanity.com/paper/2207.07605/) (2022)
- [A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME](https://mlsanity.com/paper/2305.02012/) (2023)
- [Explaining a Series of Models by Propagating Shapley Values](https://mlsanity.com/paper/2105.00108/) (2021)
- [Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search](https://mlsanity.com/paper/2206.09811/) (2022)
- [Adaptive Bias Correction for Improved Subseasonal Forecasting](https://mlsanity.com/paper/2209.10666/) (2022)
- [A Comparative Study of Methods for Estimating Conditional Shapley Values and When to Use Them](https://mlsanity.com/paper/2305.09536/) (2023)
- [Deletion and Insertion Tests in Regression Models](https://mlsanity.com/paper/2205.12423/) (2022)
- [Cohort Shapley value for algorithmic fairness](https://mlsanity.com/paper/2105.07168/) (2021)
- [Decorrelated Variable Importance](https://mlsanity.com/paper/2111.10853/) (2021)
- [Why do explanations fail? A typology and discussion on failures in XAI](https://mlsanity.com/paper/2405.13474/) (2024)
- [Variable importance without impossible data](https://mlsanity.com/paper/2205.15750/) (2022)
- [Characterizing the contribution of dependent features in XAI methods](https://mlsanity.com/paper/2304.01717/) (2023)
- [Model free variable importance for high dimensional data](https://mlsanity.com/paper/2211.08414/) (2022)
- [Beyond Demographic Parity: Redefining Equal Treatment](https://mlsanity.com/paper/2303.08040/) (2023)
- [Derivative-based Shapley value for global sensitivity analysis and machine learning explainability](https://mlsanity.com/paper/2303.15183/) (2023)
- [RbX: Region-based explanations of prediction models](https://mlsanity.com/paper/2210.08721/) (2022)
- [Error Analysis of Shapley Value-Based Model Explanations: An Informative Perspective](https://mlsanity.com/paper/2404.13522/) (2024)
- [DNN2LR: Automatic Feature Crossing for Credit Scoring](https://mlsanity.com/paper/2102.12036/) (2021)
- [What makes you unique?](https://mlsanity.com/paper/2105.08013/) (2021)
- [Confounder Analysis in Measuring Representation in Product Funnels](https://mlsanity.com/paper/2206.02962/) (2022)
- [Shapley Computations Using Surrogate Model-Based Trees](https://mlsanity.com/paper/2207.05214/) (2022)
- [Fast Approximation of the Shapley Values Based on Order-of-Addition Experimental Designs](https://mlsanity.com/paper/2309.08923/) (2023)
- [Notes on Applicability of Explainable AI Methods to Machine Learning Models Using Features Extracted by Persistent Homology](https://mlsanity.com/paper/2310.09780/) (2023)
- [The Safety Challenges of Deep Learning in Real-World Type 1 Diabetes Management](https://mlsanity.com/paper/2310.14743/) (2023)
- [Explainable Artificial Intelligence and Multicollinearity : A Mini Review of Current Approaches](https://mlsanity.com/paper/2406.11524/) (2024)
- [Explainable Artificial Intelligence for Dependent Features: Additive Effects of Collinearity](https://mlsanity.com/paper/2411.00846/) (2024)

## Abstract (arXiv)

We introduce a variable importance measure to quantify the impact of individual input variables to a black box function. Our measure is based on the Shapley value from cooperative game theory. Many measures of variable importance operate by changing some predictor values with others held fixed, potentially creating unlikely or even logically impossible combinations. Our cohort Shapley measure uses only observed data points. Instead of changing the value of a predictor we include or exclude subjects similar to the target subject on that predictor to form a similarity cohort. Then we apply Shapley value to the cohort averages. We connect variable importance measures from explainable AI to function decompositions from global sensitivity analysis. We introduce a squared cohort Shapley value that splits previously studied Shapley effects over subjects, consistent with a Shapley axiom.