# Information bottleneck through variational glasses

arXiv: https://arxiv.org/abs/1912.00830 · HTML: https://mlsanity.com/paper/1912.00830/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2019.

## Built on by (most cited first, 19 of 19)

- [CaloFlow: Fast and Accurate Generation of Calorimeter Showers with Normalizing Flows](https://mlsanity.com/paper/2106.05285/) (2021)
- [Vector Quantized Wasserstein Auto-Encoder](https://mlsanity.com/paper/2302.05917/) (2023)
- [Deep Generative Models for Detector Signature Simulation: A Taxonomic Review](https://mlsanity.com/paper/2312.09597/) (2023)
- [New directions for surrogate models and differentiable programming for High Energy Physics detector simulation](https://mlsanity.com/paper/2203.08806/) (2022)
- [Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility](https://mlsanity.com/paper/2207.04895/) (2022)
- [Glocal Information Bottleneck for Time Series Imputation](https://mlsanity.com/paper/2510.04910/) (2025)
- [Revisiting Multivariate Time Series Forecasting with Missing Values](https://mlsanity.com/paper/2509.23494/) (2025)
- [Machine learning attack on copy detection patterns: are 1x1 patterns cloneable?](https://mlsanity.com/paper/2110.02176/) (2021)
- [From Observations to States: Latent Time Series Forecasting](https://mlsanity.com/paper/2602.00297/) (2026)
- [Turbo-Sim: a generalised generative model with a physical latent space](https://mlsanity.com/paper/2112.10629/) (2021)
- [Mobile authentication of copy detection patterns](https://mlsanity.com/paper/2203.02397/) (2022)
- [Causal Information Bottleneck Boosts Adversarial Robustness of Deep Neural Network](https://mlsanity.com/paper/2210.14229/) (2022)
- [Span-based Named Entity Recognition by Generating and Compressing Information](https://mlsanity.com/paper/2302.05392/) (2023)
- [TURBO: The Swiss Knife of Auto-Encoders](https://mlsanity.com/paper/2311.06527/) (2023)
- [Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extraction](https://mlsanity.com/paper/2308.08739/) (2023)
- [Deep Privacy Funnel Model: From a Discriminative to a Generative Approach with an Application to Face Recognition](https://mlsanity.com/paper/2404.02696/) (2024)
- [Blocked and Hierarchical Disentangled Representation From Information Theory Perspective](https://mlsanity.com/paper/2101.08408/) (2021)
- [Data-driven detector signal characterization with constrained bottleneck autoencoders](https://mlsanity.com/paper/2203.04604/) (2022)
- [MIND: Lightweight and Effective Memory Injection Defense for LLM Agents via Intent-Aware Information Bottleneck](https://mlsanity.com/paper/2607.28103/) (2026)

## Abstract (arXiv)

Information bottleneck (IB) principle [1] has become an important element in information-theoretic analysis of deep models. Many state-of-the-art generative models of both Variational Autoencoder (VAE) [2; 3] and Generative Adversarial Networks (GAN) [4] families use various bounds on mutual information terms to introduce certain regularization constraints [5; 6; 7; 8; 9; 10]. Accordingly, the main difference between these models consists in add regularization constraints and targeted objectives. In this work, we will consider the IB framework for three classes of models that include supervised, unsupervised and adversarial generative models. We will apply a variational decomposition leading a common structure and allowing easily establish connections between these models and analyze underlying assumptions. Based on these results, we focus our analysis on unsupervised setup and reconsider the VAE family. In particular, we present a new interpretation of VAE family based on the IB framework using a direct decomposition of mutual information terms and show some interesting connections to existing methods such as VAE [2; 3], beta-VAE [11], AAE [12], InfoVAE [5] and VAE/GAN [13]. Instead of adding regularization constraints to an evidence lower bound (ELBO) [2; 3], which itself is a lower bound, we show that many known methods can be considered as a product of variational decomposition of mutual information terms in the IB framework. The proposed decomposition might also contribute to the interpretability of generative models of both VAE and GAN families and create a new insights to a generative compression [14; 15; 16; 17]. It can also be of interest for the analysis of novelty detection based on one-class classifiers [18] with the IB based discriminators.