# Deterministic Decoding for Discrete Data in Variational Autoencoders

arXiv: https://arxiv.org/abs/2003.02174 · HTML: https://mlsanity.com/paper/2003.02174/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 6 of 6)

- [SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantization](https://mlsanity.com/paper/2205.07547/) (2022)
- [MolGrow: A Graph Normalizing Flow for Hierarchical Molecular Generation](https://mlsanity.com/paper/2106.05856/) (2021)
- [A Variational Autoencoder for Heterogeneous Temporal and Longitudinal Data](https://mlsanity.com/paper/2204.09369/) (2022)
- [Negative Binomial Variational Autoencoders for Overdispersed Latent Modeling](https://mlsanity.com/paper/2508.05423/) (2025)
- [On the Adversarial Robustness of Generative Autoencoders in the Latent Space](https://mlsanity.com/paper/2307.02202/) (2023)
- [Likelihood-Free Variational Autoencoders](https://mlsanity.com/paper/2504.17622/) (2025)

## Abstract (arXiv)

Variational autoencoders are prominent generative models for modeling discrete data. However, with flexible decoders, they tend to ignore the latent codes. In this paper, we study a VAE model with a deterministic decoder (DD-VAE) for sequential data that selects the highest-scoring tokens instead of sampling. Deterministic decoding solely relies on latent codes as the only way to produce diverse objects, which improves the structure of the learned manifold. To implement DD-VAE, we propose a new class of bounded support proposal distributions and derive Kullback-Leibler divergence for Gaussian and uniform priors. We also study a continuous relaxation of deterministic decoding objective function and analyze the relation of reconstruction accuracy and relaxation parameters. We demonstrate the performance of DD-VAE on multiple datasets, including molecular generation and optimization problems.