# Quantifying Feature Contributions to Overall Disparity Using Information Theory

arXiv: https://arxiv.org/abs/2206.08454 · HTML: https://mlsanity.com/paper/2206.08454/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Evaluating and Aggregating Feature-based Model Explanations](https://mlsanity.com/paper/2005.00631/) (2020)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [Neuron Shapley: Discovering the Responsible Neurons](https://mlsanity.com/paper/2002.09815/) (2020)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Auditing Black-box Models for Indirect Influence](https://mlsanity.com/paper/1602.07043/) (2016)
- [CCMI : Classifier based Conditional Mutual Information Estimation](https://mlsanity.com/paper/1906.01824/) (2019)
- [Explaining Algorithmic Fairness Through Fairness-Aware Causal Path Decomposition](https://mlsanity.com/paper/2108.05335/) (2021)
- [Fairness Under Feature Exemptions: Counterfactual and Observational Measures](https://mlsanity.com/paper/2006.07986/) (2020)
- [Estimating informativeness of samples with Smooth Unique Information](https://mlsanity.com/paper/2101.06640/) (2021)
- [Explainable Fairness in Recommendation](https://mlsanity.com/paper/2204.11159/) (2022)
- [To Split or Not to Split: The Impact of Disparate Treatment in Classification](https://mlsanity.com/paper/2002.04788/) (2020)
- [Algorithmic Decision Making with Conditional Fairness](https://mlsanity.com/paper/2006.10483/) (2020)
- [Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programs](https://mlsanity.com/paper/1705.07807/) (2017)
- [Can Information Flows Suggest Targets for Interventions in Neural Circuits?](https://mlsanity.com/paper/2111.05299/) (2021)
- [Information Theoretic Measures for Fairness-aware Feature Selection](https://mlsanity.com/paper/2106.00772/) (2021)
- [Information Flow in Computational Systems](https://mlsanity.com/paper/1902.02292/) (2019)

## Built on by (most cited first, 2 of 2)

- [Multimodal Transformer for Parallel Concatenated Variational Autoencoders](https://mlsanity.com/paper/2210.16174/) (2022)
- [REFRESH: Responsible and Efficient Feature Reselection Guided by SHAP Values](https://mlsanity.com/paper/2403.08880/) (2024)

## Abstract (arXiv)

When a machine-learning algorithm makes biased decisions, it can be helpful to understand the sources of disparity to explain why the bias exists. Towards this, we examine the problem of quantifying the contribution of each individual feature to the observed disparity. If we have access to the decision-making model, one potential approach (inspired from intervention-based approaches in explainability literature) is to vary each individual feature (while keeping the others fixed) and use the resulting change in disparity to quantify its contribution. However, we may not have access to the model or be able to test/audit its outputs for individually varying features. Furthermore, the decision may not always be a deterministic function of the input features (e.g., with human-in-the-loop). For these situations, we might need to explain contributions using purely distributional (i.e., observational) techniques, rather than interventional. We ask the question: what is the "potential" contribution of each individual feature to the observed disparity in the decisions when the exact decision-making mechanism is not accessible? We first provide canonical examples (thought experiments) that help illustrate the difference between distributional and interventional approaches to explaining contributions, and when either is better suited. When unable to intervene on the inputs, we quantify the "redundant" statistical dependency about the protected attribute that is present in both the final decision and an individual feature, by leveraging a body of work in information theory called Partial Information Decomposition. We also perform a simple case study to show how this technique could be applied to quantify contributions.