# Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic

arXiv: https://arxiv.org/abs/2008.09535 · HTML: https://mlsanity.com/paper/2008.09535/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Aug 2020.

## Built on by (most cited first, 15 of 15)

- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [A Rigorous Information-Theoretic Definition of Redundancy and Relevancy in Feature Selection Based on (Partial) Information Decomposition](https://mlsanity.com/paper/2105.04187/) (2021)
- [Learning from data with structured missingness](https://mlsanity.com/paper/2304.01429/) (2023)
- [A Measure of the Complexity of Neural Representations based on Partial Information Decomposition](https://mlsanity.com/paper/2209.10438/) (2022)
- [More Synergy, Less Redundancy: Exploiting Joint Mutual Information for Self-Supervised Learning](https://mlsanity.com/paper/2307.00651/) (2023)
- [Information decomposition in complex systems via machine learning](https://mlsanity.com/paper/2307.04755/) (2023)
- [Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks](https://mlsanity.com/paper/2210.02996/) (2022)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [Ultra-marginal Feature Importance: Learning from Data with Causal Guarantees](https://mlsanity.com/paper/2204.09938/) (2022)
- [Stochastic Forward-Forward Learning through Representational Dimensionality Compression](https://mlsanity.com/paper/2505.16649/) (2025)
- [What should a neuron aim for? Designing local objective functions based on information theory](https://mlsanity.com/paper/2412.02482/) (2024)
- [Auditing Language Model Unlearning via Information Decomposition](https://mlsanity.com/paper/2601.15111/) (2026)
- [Interaction-Aware Sensitivity Analysis for Aerodynamic Optimization Results using Information Theory](https://mlsanity.com/paper/2112.05609/) (2021)
- [Rethinking Self-Supervised Learning Within the Framework of Partial Information Decomposition](https://mlsanity.com/paper/2412.02121/) (2024)
- [Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks](https://mlsanity.com/paper/2511.02584/) (2025)

## Abstract (arXiv)

Partial information decomposition (PID) seeks to decompose the multivariate mutual information that a set of source variables contains about a target variable into basic pieces, the so called "atoms of information". Each atom describes a distinct way in which the sources may contain information about the target. In this paper we show, first, that the entire theory of partial information decomposition can be derived from considerations of elementary parthood relationships between information contributions. This way of approaching the problem has the advantage of directly characterizing the atoms of information, instead of taking an indirect approach via the concept of redundancy. Secondly, we describe several intriguing links between PID and formal logic. In particular, we show how to define a measure of PID based on the information provided by certain statements about source realizations. Furthermore, we show how the mathematical lattice structure underlying PID theory can be translated into an isomorphic structure of logical statements with a particularly simple ordering relation: logical implication. The conclusion to be drawn from these considerations is that there are three isomorphic "worlds" of partial information decomposition, i.e. three equivalent ways to mathematically describe the decomposition of the information carried by a set of sources about a target: the world of parthood relationships, the world of logical statements, and the world of antichains that was utilized by Williams and Beer in their original exposition of PID theory. We additionally show how the parthood perspective provides a systematic way to answer a type of question that has been much discussed in the PID field: whether a partial information decomposition can be uniquely determined based on concepts other than redundant information.