# DVAE#: Discrete Variational Autoencoders with Relaxed Boltzmann Priors

arXiv: https://arxiv.org/abs/1805.07445 · HTML: https://mlsanity.com/paper/1805.07445/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2018.

## Built on by (most cited first, 20 of 20)

- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [GIVT: Generative Infinite-Vocabulary Transformers](https://mlsanity.com/paper/2312.02116/) (2023)
- [Controllable and Compositional Generation with Latent-Space Energy-Based Models](https://mlsanity.com/paper/2110.10873/) (2021)
- [Discrete Representations Strengthen Vision Transformer Robustness](https://mlsanity.com/paper/2111.10493/) (2021)
- [Lossy Image Compression with Quantized Hierarchical VAEs](https://mlsanity.com/paper/2208.13056/) (2022)
- [QARV: Quantization-Aware ResNet VAE for Lossy Image Compression](https://mlsanity.com/paper/2302.08899/) (2023)
- [Classical Planning in Deep Latent Space](https://mlsanity.com/paper/2107.00110/) (2021)
- [DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents](https://mlsanity.com/paper/2407.03300/) (2024)
- [Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments](https://mlsanity.com/paper/2311.04453/) (2023)
- [On the Role of Discrete Tokenization in Visual Representation Learning](https://mlsanity.com/paper/2407.09087/) (2024)
- [Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models](https://mlsanity.com/paper/2410.22775/) (2024)
- [MoFM: A Large-Scale Human Motion Foundation Model](https://mlsanity.com/paper/2502.05432/) (2025)
- [Discrete Auto-regressive Variational Attention Models for Text Modeling](https://mlsanity.com/paper/2106.08571/) (2021)
- [Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification](https://mlsanity.com/paper/2305.05871/) (2023)
- [VTD-CLIP: Video-to-Text Discretization via Prompting CLIP](https://mlsanity.com/paper/2503.18407/) (2025)
- [Anomaly Detection in Aeronautics Data with Quantum-compatible Discrete Deep Generative Model](https://mlsanity.com/paper/2303.12302/) (2023)
- [Zero-Variance Gradients for Variational Autoencoders](https://mlsanity.com/paper/2508.03587/) (2025)
- [EXCODER: EXplainable Classification Of DiscretE time series Representations](https://mlsanity.com/paper/2602.13087/) (2026)
- [Improved Variational Inference in Discrete VAEs using Error Correcting Codes](https://mlsanity.com/paper/2410.07840/) (2024)
- [Point Cloud Quantization through Multimodal Prompting for 3D Understanding](https://mlsanity.com/paper/2511.12079/) (2025)

## Abstract (arXiv)

Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches for relaxing Boltzmann machines to continuous distributions that permit training with importance-weighted bounds. These relaxations are based on generalized overlapping transformations and the Gaussian integral trick. Experiments on the MNIST and OMNIGLOT datasets show that these relaxations outperform previous discrete VAEs with Boltzmann priors. An implementation which reproduces these results is available at https://github.com/QuadrantAI/dvae .