mlsanityData to 14 Oct 2026

Learning variational autoencoders via MCMC speed measures

arXiv 2308.13731 ↗First posted 26 Aug 2023stat.ML cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Variational autoencoders (VAEs) are popular likelihood-based generative models which can be efficiently trained by maximizing an Evidence Lower Bound (ELBO). There has been much progress in improving the expressiveness of the variational distribution to obtain tighter variational bounds and increased generative performance. Whilst previous work has leveraged Markov chain Monte Carlo (MCMC) methods for the construction of variational densities, gradient-based methods for adapting the proposal distributions for deep latent variable models have received less attention. This work suggests an entropy-based adaptation for a short-run Metropolis-adjusted Langevin (MALA) or Hamiltonian Monte Carlo (HMC) chain while optimising a tighter variational bound to the log-evidence. Experiments show that this approach yields higher held-out log-likelihoods as well as improved generative metrics. Our implicit variational density can adapt to complicated posterior geometries of latent hierarchical representations arising in hierarchical VAEs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 60 of its 99 references that are in the map. First posted on arXiv 26 Aug 2023.

Builds on

60 of 99 references.

References · 60

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2015Variational Inference with Normalizing Flowscited by 2,170
  7. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
  8. 2018Demystifying MMD GANscited by 1,293
  9. 2021Variational Diffusion Modelscited by 1,153
  10. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  11. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  12. 2021Score-based Generative Modeling in Latent Spacecited by 636
38 more
  1. 2015Importance Weighted Autoencoderscited by 456
  2. 2018Invertible Residual Networkscited by 365
  3. 2016Ladder Variational Autoencoderscited by 318
  4. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 264
  5. 2019Residual Flows for Invertible Generative Modelingcited by 257
  6. 2017VAE with a VampPriorcited by 248
  7. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 233
  8. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 220
  9. 2016Variational Deep Embedding: An Unsupervised and Generative Approach to Clusteringcited by 220
  10. 2020Stochastic Normalizing Flowscited by 157
  11. 2019Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapsecited by 121
  12. 2018Log-concave sampling: Metropolis-Hastings algorithms are fastcited by 118
  13. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  14. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  15. 2015Hierarchical Variational Modelscited by 89
  16. 2017Convergence of Langevin MCMC in KL-divergencecited by 88
  17. 2019NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transportcited by 74
  18. 2020A Contrastive Learning Approach for Training Variational Autoencoder Priorscited by 66
  19. 2020Optimal dimension dependence of the Metropolis-Adjusted Langevin Algorithmcited by 57
  20. 2019Learning Hierarchical Priors in VAEscited by 53
  21. 2017Generalizing Hamiltonian Monte Carlo with Neural Networkscited by 52
  22. 2021Minimax Mixing Time of the Metropolis-Adjusted Langevin Algorithm for Log-Concave Samplingcited by 44
  23. 2018Hamiltonian Variational Auto-Encodercited by 41
  24. 2015Black-box α-divergence Minimizationcited by 37
  25. 2018Distribution Matching in Variational Inferencecited by 35
  26. 2021Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpointcited by 32
  27. 2023Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusionscited by 29
  28. 2017Geometric integrators and the Hamiltonian Monte Carlo methodcited by 25
  29. 2016Boosting Variational Inferencecited by 22
  30. 2019A Contrastive Divergence for Combining Variational Inference and MCMCcited by 21
  31. 2020On the Difficulty of Unbiased Alpha Divergence Minimizationcited by 21
  32. 2016On the Geometric Ergodicity of Hamiltonian Monte Carlocited by 19
  33. 2018Doubly Semi-Implicit Variational Inferencecited by 19
  34. 2021Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributionscited by 19
  35. 2022Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlocited by 19
  36. 2016Variational Inference with Hamiltonian Monte Carlocited by 17
  37. 2018Boosting Black Box Variational Inferencecited by 14
  38. 2019Gradient-based Adaptive Markov Chain Monte Carlocited by 14

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.