mlsanityData to 7 Oct 2026

Monte Carlo Variational Auto-Encoders

arXiv 2106.15921 ↗First posted 30 Jun 2021stat.ML cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Variational auto-encoders (VAE) are popular deep latent variable models which are trained by maximizing an Evidence Lower Bound (ELBO). To obtain tighter ELBO and hence better variational approximations, it has been proposed to use importance sampling to get a lower variance estimate of the evidence. However, importance sampling is known to perform poorly in high dimensions. While it has been suggested many times in the literature to use more sophisticated algorithms such as Annealed Importance Sampling (AIS) and its Sequential Importance Sampling (SIS) extensions, the potential benefits brought by these advanced techniques have never been realized for VAE: the AIS estimate cannot be easily differentiated, while SIS requires the specification of carefully chosen backward Markov kernels. In this paper, we address both issues and demonstrate the performance of the resulting Monte Carlo VAEs on a variety of applications.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 22 of its 36 references that are in the map. First posted on arXiv 30 Jun 2021.

Builds on

22 of 36 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015Variational Inference with Normalizing Flowscited by 2,161
  5. 2016Density estimation using Real NVPcited by 2,029
  6. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  7. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  8. 2015Importance Weighted Autoencoderscited by 454
  9. 2020Stochastic Normalizing Flowscited by 156
  10. 2017Filtering Variational Objectivescited by 91
  11. 2015Hierarchical Variational Modelscited by 89
  12. 2016Variational inference for Monte Carlo objectivescited by 83
10 more
  1. 2016Auxiliary Deep Generative Modelscited by 72
  2. 2018Importance Weighting and Variational Inferencecited by 55
  3. 2018Hamiltonian Variational Auto-Encodercited by 41
  4. 2016On the Quantitative Analysis of Decoder-Based Generative Modelscited by 40
  5. 2017Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Netcited by 30
  6. 2018Improving Explorability in Variational Inference with Annealed Variational Objectivescited by 24
  7. 2015Sandwiching the marginal likelihood using bidirectional Monte Carlocited by 20
  8. 2019Learning Deep Generative Models with Annealed Importance Samplingcited by 8
  9. 2020Unbiased Gradient Estimation for Variational Auto-Encoders using Coupled Markov Chainscited by 7
  10. 2020Nonreversible MCMC from conditional invertible transforms: a complete recipe with convergence guaranteescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 6; 2024: 6; 2025: 7; 2026: 1.

36 papers.

Sort:

By area:

  1. 2023Denoising Diffusion Samplerscited by 144
  2. 2024Iterated Denoising Energy Matching for Sampling from Boltzmann Densitiescited by 113
  3. 2022Continual Repeated Annealed Flow Transport Monte Carlocited by 66
  4. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 55
  5. 2022Score-Based Diffusion meets Annealed Importance Samplingcited by 51
  6. 2021MCMC Variational Inference via Uncorrected Hamiltonian Annealingcited by 39
  7. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  8. 2021Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpointcited by 32
  9. 2025Underdamped Diffusion Bridges with Applications to Samplingcited by 28
  10. 2024From Risk to Uncertainty: Generating Predictive Uncertainty Measures via Bayesian Estimationcited by 27
  11. 2022Langevin Diffusion Variational Inferencecited by 26
  12. 2024Liouville Flow Importance Samplercited by 25
24 more
  1. 2021Generalized Normalizing Flows via Markov Chainscited by 23
  2. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 22
  3. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  4. 2022Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlocited by 19
  5. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  6. 2021Surrogate Likelihoods for Variational Annealed Importance Samplingcited by 14
  7. 2022Multiple Importance Sampling ELBO and Deep Ensembles of Variational Approximationscited by 12
  8. 2022Bayesian inference via sparse Hamiltonian flowscited by 12
  9. 2022MixFlows: principled variational inference via mixed flowscited by 11
  10. 2024Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamicscited by 10
  11. 2023Adaptive Annealed Importance Sampling with Constant Rate Progresscited by 7
  12. 2022Cooperation in the Latent Space: The Benefits of Adding Mixture Components in Variational Autoencoderscited by 6
  13. 2023Embracing the chaos: analysis and diagnosis of numerical instability in variational flowscited by 6
  14. 2023Differentiating Metropolis-Hastings to Optimize Intractable Densitiescited by 5
  15. 2022Variational Inference with Locally Enhanced Bounds for Hierarchical Modelscited by 4
  16. 2025End-To-End Learning of Gaussian Mixture Priors for Diffusion Samplercited by 4
  17. 2025Model-Informed Flows for Bayesian Inferencecited by 3
  18. 2023Reparameterized Variational Rejection Samplingcited by 2
  19. 2024Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distributioncited by 2
  20. 2025Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coveragecited by 2
  21. 2022Optimization of Annealed Importance Sampling Hyperparameterscited by 1
  22. 2025Asymptotically exact variational flows via involutive MCMC kernelscited by 1
  23. 2025Continuously Tempered Diffusion Samplerscited by 1
  24. 2026Quasi Monte Carlo methods enable extremely low-dimensional deep generative modelscited by 0