# On the Quantitative Analysis of Decoder-Based Generative Models

arXiv: https://arxiv.org/abs/1611.04273 · HTML: https://mlsanity.com/paper/1611.04273/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2016.

## Built on by (most cited first, 30 of 40)

- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [On Memorization in Probabilistic Deep Generative Models](https://mlsanity.com/paper/2106.03216/) (2021)
- [Machine Unlearning for Image-to-Image Generative Models](https://mlsanity.com/paper/2402.00351/) (2024)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Feature Likelihood Divergence: Evaluating the Generalization of Generative Models Using Samples](https://mlsanity.com/paper/2302.04440/) (2023)
- [Generative Modeling for Multi-task Visual Learning](https://mlsanity.com/paper/2106.13409/) (2021)
- [Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills](https://mlsanity.com/paper/2309.13614/) (2023)
- [q-Paths: Generalizing the Geometric Annealing Path using Power Means](https://mlsanity.com/paper/2107.00745/) (2021)
- [ProbNeRF: Uncertainty-Aware Inference of 3D Shapes from 2D Images](https://mlsanity.com/paper/2210.17415/) (2022)
- [Deterministic Langevin Monte Carlo with Normalizing Flows for Bayesian Inference](https://mlsanity.com/paper/2205.14240/) (2022)
- [Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport](https://mlsanity.com/paper/2202.01841/) (2022)
- [Compositional diversity in visual concept learning](https://mlsanity.com/paper/2305.19374/) (2023)
- [Uncertainty-Aware Decision Transformer for Stochastic Driving Environments](https://mlsanity.com/paper/2309.16397/) (2023)
- [Adaptive Annealed Importance Sampling with Constant Rate Progress](https://mlsanity.com/paper/2306.15283/) (2023)
- [Interpreting intermediate convolutional layers of generative CNNs trained on waveforms](https://mlsanity.com/paper/2104.09489/) (2021)
- [Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoders](https://mlsanity.com/paper/2209.15514/) (2022)
- [Matching aggregate posteriors in the variational autoencoder](https://mlsanity.com/paper/2311.07693/) (2023)
- [SeriesGAN: Time Series Generation via Adversarial and Autoregressive Learning](https://mlsanity.com/paper/2410.21203/) (2024)
- [Bootstrap Your Flow](https://mlsanity.com/paper/2111.11510/) (2021)
- [Fast-NTK: Parameter-Efficient Unlearning for Large-Scale Models](https://mlsanity.com/paper/2312.14923/) (2023)
- [Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey](https://mlsanity.com/paper/2111.13282/) (2021)
- [Variational Inference with Locally Enhanced Bounds for Hierarchical Models](https://mlsanity.com/paper/2203.04432/) (2022)
- [Diffusing Gaussian Mixtures for Generating Categorical Data](https://mlsanity.com/paper/2303.04635/) (2023)
- [Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling](https://mlsanity.com/paper/2408.06710/) (2024)
- [Variational Inference with Holder Bounds](https://mlsanity.com/paper/2111.02947/) (2021)
- [Estimators of Entropy and Information via Inference in Probabilistic Models](https://mlsanity.com/paper/2202.12363/) (2022)
- [Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation](https://mlsanity.com/paper/2308.07645/) (2023)

## Abstract (arXiv)

The past several years have seen remarkable progress in generative models which produce convincing samples of images and other modalities. A shared component of many powerful generative models is a decoder network, a parametric deep neural net that defines a generative distribution. Examples include variational autoencoders, generative adversarial networks, and generative moment matching networks. Unfortunately, it can be difficult to quantify the performance of these models because of the intractability of log-likelihood estimation, and inspecting samples can be misleading. We propose to use Annealed Importance Sampling for evaluating log-likelihoods for decoder-based models and validate its accuracy using bidirectional Monte Carlo. The evaluation code is provided at https://github.com/tonywu95/eval_gen. Using this technique, we analyze the performance of decoder-based models, the effectiveness of existing log-likelihood estimators, the degree of overfitting, and the degree to which these models miss important modes of the data distribution.