# Variational Information Bottleneck on Vector Quantized Autoencoders

arXiv: https://arxiv.org/abs/1808.01048 · HTML: https://mlsanity.com/paper/1808.01048/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2018.

## Built on by (most cited first, 5 of 5)

- [Vector Quantized Wasserstein Auto-Encoder](https://mlsanity.com/paper/2302.05917/) (2023)
- [CQM: Curriculum Reinforcement Learning with a Quantized World Model](https://mlsanity.com/paper/2310.17330/) (2023)
- [Human-Guided Complexity-Controlled Abstractions](https://mlsanity.com/paper/2310.17550/) (2023)
- [Implicit Feature Decoupling with Depthwise Quantization](https://mlsanity.com/paper/2203.08080/) (2022)
- [Deconstructing Generative Diversity: An Information Bottleneck Analysis of Discrete Latent Generative Models](https://mlsanity.com/paper/2512.01831/) (2025)

## Abstract (arXiv)

In this paper, we provide an information-theoretic interpretation of the Vector Quantized-Variational Autoencoder (VQ-VAE). We show that the loss function of the original VQ-VAE can be derived from the variational deterministic information bottleneck (VDIB) principle. On the other hand, the VQ-VAE trained by the Expectation Maximization (EM) algorithm can be viewed as an approximation to the variational information bottleneck(VIB) principle.