# A Novel Approach to the Partial Information Decomposition

arXiv: https://arxiv.org/abs/1908.08642 · HTML: https://mlsanity.com/paper/1908.08642/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [Interpretable Diffusion via Information Decomposition](https://mlsanity.com/paper/2310.07972/) (2023)
- [Gacs-Korner Common Information Variational Autoencoder](https://mlsanity.com/paper/2205.12239/) (2022)
- [Demystifying Local and Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition](https://mlsanity.com/paper/2307.11333/) (2023)
- [Information decomposition in complex systems via machine learning](https://mlsanity.com/paper/2307.04755/) (2023)
- [Ultra-marginal Feature Importance: Learning from Data with Causal Guarantees](https://mlsanity.com/paper/2204.09938/) (2022)
- [Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition](https://mlsanity.com/paper/2606.00959/) (2026)
- [Disentangling Interactions and Dependencies in Feature Attribution](https://mlsanity.com/paper/2410.23772/) (2024)
- [Partial information decomposition: redundancy as information bottleneck](https://mlsanity.com/paper/2405.07665/) (2024)
- [Broadcast Channel Cooperative Gain: An Operational Interpretation of Partial Information Decomposition](https://mlsanity.com/paper/2502.10878/) (2025)
- [Explainable Multimodal Regression via Information Decomposition](https://mlsanity.com/paper/2512.22102/) (2025)
- [ICYM2I: The illusion of multimodal informativeness under missingness](https://mlsanity.com/paper/2505.16953/) (2025)
- [SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation](https://mlsanity.com/paper/2605.18920/) (2026)
- [The Representational Limit of Scalar Interactions: An Interventional Decomposition](https://mlsanity.com/paper/2606.19410/) (2026)
- [Which Information Matters? Dissecting Human-written Multi-document Summaries with Partial Information Decomposition](https://mlsanity.com/paper/2405.14470/) (2024)
- [Auditing Algorithmic Bias in Transformer-Based Trading](https://mlsanity.com/paper/2510.05140/) (2025)
- [Quantifying Document Impact in RAG-LLMs](https://mlsanity.com/paper/2601.05260/) (2025)
- [Gaussian BROJA Partial Information Decomposition in Closed Form](https://mlsanity.com/paper/2610.13464/) (2026)

## Abstract (arXiv)

We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.