mlsanityData to 7 Oct 2026

Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors

arXiv 2401.02739 ↗First posted 5 Jan 2024cs.LG q-bio.QM stat.MLcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our method introduces an expressive class of diffusion-based variational posteriors that perform iterative refinement in latent space; we train these posteriors with a novel regularized evidence lower bound (ELBO) on the marginal likelihood inspired by the wake-sleep algorithm. Our method is easy to implement (it fits a regularized extension of the ELBO), is compatible with black-box variational inference, and outperforms alternative classes of approximate posteriors based on normalizing flows or adversarial networks. We find that DDVI improves inference and learning in deep latent variable models across common benchmarks as well as on a motivating task in biology -- inferring latent ancestry from human genomes -- where it outperforms strong baselines on the Thousand Genomes dataset.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 41 of its 64 references that are in the map. First posted on arXiv 5 Jan 2024.

Builds on

41 of 64 references.

Closest ancestors · 14

Same direction, newest first

  1. 2023InfoDiffusion: Representation Learning Using Information Maximizing Diffusion Models14 Jun
  2. 2023Denoising Diffusion Samplers27 Feb
  3. 2022Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Models26 Dec
  4. 2022An optimal control perspective on diffusion-based generative modeling2 Nov
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  8. 2021Variational Diffusion Models1 Jul
  9. 2021Diffusion Priors In Variational Autoencoders29 Jun
  10. 2021Score-based Generative Modeling in Latent Space10 Jun
  11. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
2 more
  1. 2020Denoising Diffusion Implicit Models6 Oct
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 27

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  3. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  4. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  5. 2015Variational Inference with Normalizing Flowscited by 2,161
  6. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  7. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  8. 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
  9. 2015Adversarial Autoencoderscited by 359
  10. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  11. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  12. 2022Transformer Neural Processes: Uncertainty-Aware Meta Learning Via Sequence Modelingcited by 126
15 more
  1. 2017InfoVAE: Information Maximizing Variational Autoencoderscited by 124
  2. 2016Composing graphical models with neural networks for structured representations and fast inferencecited by 113
  3. 2023Improved sampling via learned diffusionscited by 103
  4. 2015Hierarchical Variational Modelscited by 89
  5. 2016Auxiliary Deep Generative Modelscited by 72
  6. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  7. 2022Score-Based Diffusion meets Annealed Importance Samplingcited by 51
  8. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  9. 2022Backpropagation through Combinatorial Algorithms: Identity with Projection Workscited by 29
  10. 2022Diffusion bridges vector quantized Variational AutoEncoderscited by 16
  11. 2021Autoregressive Quantile Flows for Predictive Uncertainty Estimationcited by 15
  12. 2022Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxiescited by 13
  13. 2021On Incorporating Inductive Biases into VAEscited by 11
  14. 2022Semi-Autoregressive Energy Flows: Exploring Likelihood-Free Training of Normalizing Flowscited by 9
  15. 2022Semi-Parametric Inducing Point Networks and Neural Processescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Vision-Driven Prompt Optimization for Large Language Models in Multimodal Generative Taskscited by 0
  2. 2026Mode Collapse Is Cheap to Detect: A Ground-Truth-Free Pre-Flight Check for Neural Samplerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diffusion Bridge Variational Inference for Deep Gaussian Processes
  2. 2025A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models
  3. 2024Sparse Inducing Points in Deep Gaussian Processes: Enhancing Modeling with Denoising Diffusion Variational Inference
  4. 2025Diffusion Models: A Mathematical Introduction
  5. 2022Diffusion models as plug-and-play priors
  6. 2022Understanding Diffusion Models: A Unified Perspective
  7. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  8. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Models