# Leveraging the Exact Likelihood of Deep Latent Variable Models

arXiv: https://arxiv.org/abs/1802.04826 · HTML: https://mlsanity.com/paper/1802.04826/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2018.

## Built on by (most cited first, 23 of 23)

- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [Diagnosing and Fixing Manifold Overfitting in Deep Generative Models](https://mlsanity.com/paper/2204.07172/) (2022)
- [Learning Manifold Dimensions with Conditional Variational Autoencoders](https://mlsanity.com/paper/2302.11756/) (2023)
- [Laplacian Autoencoders for Learning Stochastic Representations](https://mlsanity.com/paper/2206.15078/) (2022)
- [Tree Variational Autoencoders](https://mlsanity.com/paper/2306.08984/) (2023)
- [Bounding Evidence and Estimating Log-Likelihood in VAE](https://mlsanity.com/paper/2206.09453/) (2022)
- [Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detection](https://mlsanity.com/paper/2109.09374/) (2021)
- [Deterministic training of generative autoencoders using invertible layers](https://mlsanity.com/paper/2205.09546/) (2022)
- [Online reinforcement learning with sparse rewards through an active inference capsule](https://mlsanity.com/paper/2106.02390/) (2021)
- [Robust uncertainty estimates with out-of-distribution pseudo-inputs training](https://mlsanity.com/paper/2201.05890/) (2022)
- [Leveraging variational autoencoders for multiple data imputation](https://mlsanity.com/paper/2209.15321/) (2022)
- [FFHFlow: Diverse and Uncertainty-Aware Dexterous Grasp Generation via Flow Variational Inference](https://mlsanity.com/paper/2407.15161/) (2024)
- [Conditional Sampling of Variational Autoencoders via Iterated Approximate Ancestral Sampling](https://mlsanity.com/paper/2308.09078/) (2023)
- [Improving Variational Autoencoder Estimation from Incomplete Data with Mixture Variational Families](https://mlsanity.com/paper/2403.03069/) (2024)
- [Boundary-enhanced time series data imputation with long-term dependency diffusion models](https://mlsanity.com/paper/2501.06585/) (2025)
- [AASAE: Augmentation-Augmented Stochastic Autoencoders](https://mlsanity.com/paper/2107.12329/) (2021)
- [Benchmarking Generative Latent Variable Models for Speech](https://mlsanity.com/paper/2202.12707/) (2022)
- [Interpolation of Missing Swaption Volatility Data using Gibbs Sampling on Variational Autoencoders](https://mlsanity.com/paper/2204.10400/) (2022)
- [Adversarially Contrastive Estimation of Conditional Neural Processes](https://mlsanity.com/paper/2303.13004/) (2023)
- [Diverse super-resolution with pretrained deep hiererarchical VAEs](https://mlsanity.com/paper/2205.10347/) (2022)
- [Beta-Sigma VAE: Separating beta and decoder variance in Gaussian variational autoencoder](https://mlsanity.com/paper/2409.09361/) (2024)
- [Filling in Missing FX Implied Volatilities with Uncertainties: Improving VAE-Based Volatility Imputation](https://mlsanity.com/paper/2411.05998/) (2024)
- [Quasi Monte Carlo methods enable extremely low-dimensional deep generative models](https://mlsanity.com/paper/2601.18676/) (2026)

## Abstract (arXiv)

Deep latent variable models (DLVMs) combine the approximation abilities of deep neural networks and the statistical foundations of generative models. Variational methods are commonly used for inference; however, the exact likelihood of these models has been largely overlooked. The purpose of this work is to study the general properties of this quantity and to show how they can be leveraged in practice. We focus on important inferential problems that rely on the likelihood: estimation and missing data imputation. First, we investigate maximum likelihood estimation for DLVMs: in particular, we show that most unconstrained models used for continuous data have an unbounded likelihood function. This problematic behaviour is demonstrated to be a source of mode collapse. We also show how to ensure the existence of maximum likelihood estimates, and draw useful connections with nonparametric mixture models. Finally, we describe an algorithm for missing data imputation using the exact conditional likelihood of a deep latent variable model. On several data sets, our algorithm consistently and significantly outperforms the usual imputation scheme used for DLVMs.