# Partial information decomposition: redundancy as information bottleneck

arXiv: https://arxiv.org/abs/2405.07665 · HTML: https://mlsanity.com/paper/2405.07665/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 15 of 15)

- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Learning Robust Representations via Multi-View Information Bottleneck](https://mlsanity.com/paper/2002.07017/) (2020)
- [The Conditional Entropy Bottleneck](https://mlsanity.com/paper/2002.05379/) (2020)
- [Nonlinear Information Bottleneck](https://mlsanity.com/paper/1705.02436/) (2017)
- [The Information Bottleneck Problem and Its Applications in Machine Learning](https://mlsanity.com/paper/2004.14941/) (2020)
- [Caveats for information bottleneck in deterministic scenarios](https://mlsanity.com/paper/1808.07593/) (2018)
- [Quantifying High-order Interdependencies via Multivariate Extensions of the Mutual Information](https://mlsanity.com/paper/1902.11239/) (2019)
- [Predictive information in a sensory population](https://mlsanity.com/paper/1307.0225/) (2013)
- [Multivariate Information Bottleneck](https://mlsanity.com/paper/1301.2270/) (2013)
- [A Novel Approach to the Partial Information Decomposition](https://mlsanity.com/paper/1908.08642/) (2019)
- [The Convex Information Bottleneck Lagrangian](https://mlsanity.com/paper/1911.11000/) (2019)
- [Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions](https://mlsanity.com/paper/1510.00831/) (2015)
- [Machine-learning optimized measurements of chaotic dynamical systems via the information bottleneck](https://mlsanity.com/paper/2311.04896/) (2023)
- [The Variational Deficiency Bottleneck](https://mlsanity.com/paper/1810.11677/) (2018)
- [Fast and exact search for the partition with minimal information loss](https://mlsanity.com/paper/1708.01444/) (2017)

## Built on by (most cited first, 3 of 3)

- [A Generalized Information Bottleneck Theory of Deep Learning](https://mlsanity.com/paper/2509.26327/) (2025)
- [Source-Side Sufficiency for the Information Bottleneck: Exact Reduction and Finite-Block Equivalence](https://mlsanity.com/paper/2604.26744/) (2026)
- [Surveying the space of descriptions of a composite system with machine learning](https://mlsanity.com/paper/2411.18579/) (2024)

## Abstract (arXiv)

The partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem, termed the "redundancy bottleneck" (RB). The RB formalizes a tradeoff between prediction and compression: it extracts information from the sources that best predict the target, without revealing which source provided the information. It can be understood as a generalization of "Blackwell redundancy", which we previously proposed as a principled measure of PID redundancy. The "RB curve" quantifies the prediction--compression tradeoff at multiple scales. This curve can also be quantified for individual sources, allowing subsets of redundant sources to be identified without combinatorial optimization. We provide an efficient iterative algorithm for computing the RB curve.