# Variational Predictive Information Bottleneck

arXiv: https://arxiv.org/abs/1910.10831 · HTML: https://mlsanity.com/paper/1910.10831/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 8 of 8)

- [VJEPA: Variational Joint Embedding Predictive Architectures as Probabilistic World Models](https://mlsanity.com/paper/2601.14354/) (2026)
- [Understanding Neural Networks with Logarithm Determinant Entropy Estimator](https://mlsanity.com/paper/2105.03705/) (2021)
- [Simplifying Latent Dynamics with Softly State-Invariant World Models](https://mlsanity.com/paper/2401.17835/) (2024)
- [Separating common from salient patterns with Contrastive Representation Learning](https://mlsanity.com/paper/2402.11928/) (2024)
- [LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs](https://mlsanity.com/paper/2605.06915/) (2026)
- [Deep Reference Priors: What is the best way to pretrain a model?](https://mlsanity.com/paper/2202.00187/) (2022)
- [Federated Variational Inference: Towards Improved Personalization and Generalization](https://mlsanity.com/paper/2305.13672/) (2023)
- [Convolutional Bayesian Filtering](https://mlsanity.com/paper/2404.00481/) (2024)

## Abstract (arXiv)

In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this functional as a variational lower bound of a predictive information bottleneck objective. This generalized functional encompasses most modern inference procedures and suggests novel ones.