# Variational Information Bottleneck for Unsupervised Clustering: Deep Gaussian Mixture Embedding

arXiv: https://arxiv.org/abs/1905.11741 · HTML: https://mlsanity.com/paper/1905.11741/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2019.

## Built on by (most cited first, 8 of 8)

- [To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review](https://mlsanity.com/paper/2304.09355/) (2023)
- [Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximations](https://mlsanity.com/paper/2202.10951/) (2022)
- [TURBO: The Swiss Knife of Auto-Encoders](https://mlsanity.com/paper/2311.06527/) (2023)
- [Deep Graph Clustering via Mutual Information Maximization and Mixture Model](https://mlsanity.com/paper/2205.05168/) (2022)
- [The Wyner Variational Autoencoder for Unsupervised Multi-Layer Wireless Fingerprinting](https://mlsanity.com/paper/2303.15860/) (2023)
- [Efficient Alternating Minimization Solvers for Wyner Multi-View Unsupervised Learning](https://mlsanity.com/paper/2303.15866/) (2023)
- [A generative recommender system with GMM prior for cancer drug generation and sensitivity prediction](https://mlsanity.com/paper/2206.03555/) (2022)
- [Clustering based on Mixtures of Sparse Gaussian Processes](https://mlsanity.com/paper/2303.13665/) (2023)

## Abstract (arXiv)

In this paper, we develop an unsupervised generative clustering framework that combines the Variational Information Bottleneck and the Gaussian Mixture Model. Specifically, in our approach, we use the Variational Information Bottleneck method and model the latent space as a mixture of Gaussians. We derive a bound on the cost function of our model that generalizes the Evidence Lower Bound (ELBO) and provide a variational inference type algorithm that allows computing it. In the algorithm, the coders' mappings are parametrized using neural networks, and the bound is approximated by Monte Carlo sampling and optimized with stochastic gradient descent. Numerical results on real datasets are provided to support the efficiency of our method.