mlsanityData to 7 Oct 2026

Partial Information Decomposition for Data Interpretability and Feature Selection

arXiv 2405.19212 ↗First posted 29 May 2024cs.LG cs.AI cs.ITcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Partial Information Decomposition of Features (PIDF), a new paradigm for simultaneous data interpretability and feature selection. Contrary to traditional methods that assign a single importance value, our approach is based on three metrics per feature: the mutual information shared with the target variable, the feature's contribution to synergistic information, and the amount of this information that is redundant. In particular, we develop a novel procedure based on these three metrics, which reveals not only how features are correlated with the target but also the additional and overlapping information provided by considering them in combination with other features. We extensively evaluate PIDF using both synthetic and real-world data, demonstrating its potential applications and effectiveness, by considering case studies from genetics and neuroscience.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 62 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

22 of 62 references.

References · 22

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2019On Variational Bounds of Mutual Informationcited by 550
  3. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  4. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  5. 2016Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Modelscited by 212
  6. 2019Explaining individual predictions when features are dependent: More accurate approximations to Shapley valuescited by 205
  7. 2020Problems with Shapley-value-based explanations as feature importance measurescited by 198
  8. 2020True to the Model or True to the Data?cited by 123
  9. 2019Asymmetric Shapley values: incorporating causal knowledge into model-agnostic explainabilitycited by 114
  10. 2023Quantifying & Modeling Multimodal Interactions: An Information Decomposition Frameworkcited by 81
  11. 2023Learning to Maximize Mutual Information for Dynamic Feature Selectioncited by 54
  12. 2022WeightedSHAP: analyzing and improving Shapley based feature attributionscited by 36
10 more
  1. 2019Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Informationcited by 30
  2. 2021A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decompositioncited by 25
  3. 2021Instance-wise Causal Feature Selection for Model Interpretationcited by 8
  4. 2016Variational Information Maximization for Feature Selectioncited by 7
  5. 2018BROJA-2PID: A robust estimator for bivariate partial information decompositioncited by 6
  6. 2022Ultra-marginal Feature Importance: Learning from Data with Causal Guaranteescited by 6
  7. 2020Active Feature Selection for the Mutual Information Criterioncited by 4
  8. 2023Feature Selection in the Contrastive Analysis Settingcited by 3
  9. 2024TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learningcited by 3
  10. 2019MAXENT3D_PID: An Estimator for the Maximum-entropy Trivariate Partial Information Decompositioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025A Generalized Information Bottleneck Theory of Deep Learningcited by 4
  2. 2026Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Informationcited by 0
  3. 2026Partial Information Decomposition as a Multi-Contrast 3D MRI Selection Strategy for Resource-Constrained Deep Neural Network Training in Brain Tumor Segmentationcited by 0