# Partial Information Decomposition for Data Interpretability and Feature Selection

arXiv: https://arxiv.org/abs/2405.19212 · HTML: https://mlsanity.com/paper/2405.19212/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 62 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 22 of 22)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models](https://mlsanity.com/paper/1612.08468/) (2016)
- [Explaining individual predictions when features are dependent: More accurate approximations to Shapley values](https://mlsanity.com/paper/1903.10464/) (2019)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [True to the Model or True to the Data?](https://mlsanity.com/paper/2006.16234/) (2020)
- [Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainability](https://mlsanity.com/paper/1910.06358/) (2019)
- [Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework](https://mlsanity.com/paper/2302.12247/) (2023)
- [Learning to Maximize Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2301.00557/) (2023)
- [WeightedSHAP: analyzing and improving Shapley based feature attributions](https://mlsanity.com/paper/2209.13429/) (2022)
- [Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information](https://mlsanity.com/paper/1902.11239/) (2019)
- [A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition](https://mlsanity.com/paper/2105.04187/) (2021)
- [Instance-wise Causal Feature Selection for Model Interpretation](https://mlsanity.com/paper/2104.12759/) (2021)
- [Variational Information Maximization for Feature Selection](https://mlsanity.com/paper/1606.02827/) (2016)
- [BROJA-2PID: A robust estimator for bivariate partial information decomposition](https://mlsanity.com/paper/1802.02485/) (2018)
- [Ultra-marginal Feature Importance: Learning from Data with Causal Guarantees](https://mlsanity.com/paper/2204.09938/) (2022)
- [Active Feature Selection for the Mutual Information Criterion](https://mlsanity.com/paper/2012.06979/) (2020)
- [Feature Selection in the Contrastive Analysis Setting](https://mlsanity.com/paper/2310.18531/) (2023)
- [TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning](https://mlsanity.com/paper/2401.11512/) (2024)
- [MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decomposition](https://mlsanity.com/paper/1901.03352/) (2019)

## Built on by (most cited first, 4 of 4)

- [A Generalized Information Bottleneck Theory of Deep Learning](https://mlsanity.com/paper/2509.26327/) (2025)
- [Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information](https://mlsanity.com/paper/2605.07086/) (2026)
- [Partial Information Decomposition as a Multi-Contrast 3D MRI Selection Strategy for Resource-Constrained Deep Neural Network Training in Brain Tumor Segmentation](https://mlsanity.com/paper/2607.15396/) (2026)
- [Gaussian BROJA Partial Information Decomposition in Closed Form](https://mlsanity.com/paper/2610.13464/) (2026)

## Abstract (arXiv)

In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature's contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience.