# Discrete Variational Autoencoders

arXiv: https://arxiv.org/abs/1609.02200 · HTML: https://mlsanity.com/paper/1609.02200/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2016.

## Built on by (most cited first, 30 of 125)

- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [An Image is Worth 32 Tokens for Reconstruction and Generation](https://mlsanity.com/paper/2406.07550/) (2024)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT](https://mlsanity.com/paper/2303.04226/) (2023)
- [PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers](https://mlsanity.com/paper/2111.12710/) (2021)
- [VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling](https://mlsanity.com/paper/2111.12681/) (2021)
- [Masked Discrimination for Self-Supervised Learning on Point Clouds](https://mlsanity.com/paper/2203.11183/) (2022)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [Adaptive Caching for Faster Video Generation with Diffusion Transformers](https://mlsanity.com/paper/2411.02397/) (2024)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities](https://mlsanity.com/paper/2306.04829/) (2023)
- [MILAN: Masked Image Pretraining on Language Assisted Representation](https://mlsanity.com/paper/2208.06049/) (2022)
- [Membership Inference Attacks Against Text-to-image Generation Models](https://mlsanity.com/paper/2210.00968/) (2022)
- [Corrupted Image Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2202.03382/) (2022)
- [MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2108.09274/) (2021)
- [Hard Patches Mining for Masked Image Modeling](https://mlsanity.com/paper/2304.05919/) (2023)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Event Camera Data Pre-training](https://mlsanity.com/paper/2301.01928/) (2023)
- [MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers](https://mlsanity.com/paper/2205.13137/) (2022)
- [RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model](https://mlsanity.com/paper/2309.00810/) (2023)
- [Regularized Vector Quantization for Tokenized Image Synthesis](https://mlsanity.com/paper/2303.06424/) (2023)
- [Masked Event Modeling: Self-Supervised Pretraining for Event Cameras](https://mlsanity.com/paper/2212.10368/) (2022)
- [Recent Advances for Quantum Neural Networks in Generative Learning](https://mlsanity.com/paper/2206.03066/) (2022)
- [Self-Guided Diffusion Models](https://mlsanity.com/paper/2210.06462/) (2022)

## Abstract (arXiv)

Probabilistic models with discrete latent variables naturally capture datasets composed of discrete classes. However, they are difficult to train efficiently, since backpropagation through discrete variables is generally not possible. We present a novel method to train a class of probabilistic models with discrete latent variables using the variational autoencoder framework, including backpropagation through the discrete latent variables. The associated class of probabilistic models comprises an undirected discrete component and a directed hierarchical continuous component. The discrete component captures the distribution over the disconnected smooth manifolds induced by the continuous component. As a result, this class of models efficiently learns both the class of objects in an image, and their specific realization in pixels, from unsupervised data, and outperforms state-of-the-art methods on the permutation-invariant MNIST, Omniglot, and Caltech-101 Silhouettes datasets.