# Multivariate Information Bottleneck

arXiv: https://arxiv.org/abs/1301.2270 · HTML: https://mlsanity.com/paper/1301.2270/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jan 2013.

## Built on by (most cited first, 17 of 17)

- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning](https://mlsanity.com/paper/2404.09403/) (2024)
- [Deep Variational Multivariate Information Bottleneck -- A Framework for Variational Losses](https://mlsanity.com/paper/2310.03311/) (2023)
- [Simultaneous Dimensionality Reduction: A Data Efficient Approach for Multimodal Representations Learning](https://mlsanity.com/paper/2310.04458/) (2023)
- [Data efficiency, dimensionality reduction, and the generalized symmetric information bottleneck](https://mlsanity.com/paper/2309.05649/) (2023)
- [A Generalized Information Bottleneck Theory of Deep Learning](https://mlsanity.com/paper/2509.26327/) (2025)
- [Disentangled Representation Learning with Transmitted Information Bottleneck](https://mlsanity.com/paper/2311.01686/) (2023)
- [Partial information decomposition: redundancy as information bottleneck](https://mlsanity.com/paper/2405.07665/) (2024)
- [Accurate Estimation of Mutual Information in High Dimensional Data](https://mlsanity.com/paper/2506.00330/) (2025)
- [Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning](https://mlsanity.com/paper/2606.07770/) (2026)
- [Scalable Vector Gaussian Information Bottleneck](https://mlsanity.com/paper/2102.07525/) (2021)
- [RegCLR: A Self-Supervised Framework for Tabular Representation Learning in the Wild](https://mlsanity.com/paper/2211.01165/) (2022)
- [Elastic Information Bottleneck](https://mlsanity.com/paper/2311.03955/) (2023)
- [Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data](https://mlsanity.com/paper/2604.24662/) (2026)
- [An information-Theoretic Approach to Semi-supervised Transfer Learning](https://mlsanity.com/paper/2306.06731/) (2023)
- [Neuro-Inspired Hierarchical Multimodal Learning](https://mlsanity.com/paper/2309.15877/) (2023)
- [Causally-Aware Information Bottleneck for Domain Adaptation](https://mlsanity.com/paper/2601.04361/) (2026)

## Abstract (arXiv)

The Information bottleneck method is an unsupervised non-parametric data organization technique. Given a joint distribution P(A,B), this method constructs a new variable T that extracts partitions, or clusters, over the values of A that are informative about B. The information bottleneck has already been applied to document classification, gene expression, neural code, and spectral analysis. In this paper, we introduce a general principled framework for multivariate extensions of the information bottleneck method. This allows us to consider multiple systems of data partitions that are inter-related. Our approach utilizes Bayesian networks for specifying the systems of clusters and what information each captures. We show that this construction provides insight about bottleneck variations and enables us to characterize solutions of these variations. We also present a general framework for iterative algorithms for constructing solutions, and apply it to several examples.