# Bounding Evidence and Estimating Log-Likelihood in VAE

arXiv: https://arxiv.org/abs/2206.09453 · HTML: https://mlsanity.com/paper/2206.09453/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 37 references that are in the map.
- First posted on arXiv 19 Jun 2022.

## Builds on (most cited first, 14 of 14)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Tighter Variational Bounds are Not Necessarily Better](https://mlsanity.com/paper/1802.04537/) (2018)
- [The Thermodynamic Variational Objective](https://mlsanity.com/paper/1907.00031/) (2019)
- [Leveraging the Exact Likelihood of Deep Latent Variable Models](https://mlsanity.com/paper/1802.04826/) (2018)
- [Sandwiching the marginal likelihood using bidirectional Monte Carlo](https://mlsanity.com/paper/1511.02543/) (2015)
- [Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models](https://mlsanity.com/paper/2101.07046/) (2021)
- [Stochastic Variational Inference via Upper Bound](https://mlsanity.com/paper/1912.00650/) (2019)
- [Improving Actor-Critic Reinforcement Learning via Hamiltonian Monte Carlo Method](https://mlsanity.com/paper/2103.12020/) (2021)

## Built on by (most cited first, 6 of 6)

- [Are Ensembles Getting Better all the Time?](https://mlsanity.com/paper/2311.17885/) (2023)
- [Forward chi2 Divergence Based Variational Importance Sampling](https://mlsanity.com/paper/2311.02516/) (2023)
- [TUBE: Tangent Upper Bound on Evidence for Discrete Diffusion Language Models](https://mlsanity.com/paper/2605.24292/) (2026)
- [A Conditional Diffusion Model for Electrical Impedance Tomography Image Reconstruction](https://mlsanity.com/paper/2412.16979/) (2024)
- [Tight Bounds for Jensen's Gap with Applications to Variational Inference](https://mlsanity.com/paper/2502.03988/) (2025)
- [Quasi Monte Carlo methods enable extremely low-dimensional deep generative models](https://mlsanity.com/paper/2601.18676/) (2026)

## Abstract (arXiv)

Many crucial problems in deep learning and statistical inference are caused by a variational gap, i.e., a difference between model evidence (log-likelihood) and evidence lower bound (ELBO). In particular, in a classical VAE setting that involves training via an ELBO cost function, it is difficult to provide a robust comparison of the effects of training between models, since we do not know a log-likelihood of data (but only its lower bound). In this paper, to deal with this problem, we introduce a general and effective upper bound, which allows us to efficiently approximate the evidence of data. We provide extensive theoretical and experimental studies of our approach, including its comparison to the other state-of-the-art upper bounds, as well as its application as a tool for the evaluation of models that were trained on various lower bounds.