mlsanityData to 7 Oct 2026

Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling

arXiv 2406.07423 ↗First posted 11 Jun 2024cs.LG cs.AI stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

Monte Carlo methods, Variational Inference, and their combinations play a pivotal role in sampling from intractable probability distributions. However, current studies lack a unified evaluation framework, relying on disparate performance measures and limited method comparisons across diverse tasks, complicating the assessment of progress and hindering the decision-making of practitioners. In response to these challenges, our work introduces a benchmark that evaluates sampling methods using a standardized task suite and a broad range of performance criteria. Moreover, we study existing metrics for quantifying mode collapse and introduce novel metrics for this purpose. Our findings provide insights into strengths and weaknesses of existing sampling methods, serving as a valuable reference for future developments. The code is publicly available here.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 37 of its 70 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

37 of 70 references.

References · 37

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2016Improved Techniques for Training GANscited by 2,938
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2016Density estimation using Real NVPcited by 2,029
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  7. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  8. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  9. 2019Neural Spline Flowscited by 590
  10. 2013A survey of the Schr"odinger problem and some of its connections with optimal transportcited by 286
  11. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  12. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
25 more
  1. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  2. 2020Stochastic Normalizing Flowscited by 156
  3. 2022An optimal control perspective on diffusion-based generative modelingcited by 145
  4. 2023Denoising Diffusion Samplerscited by 144
  5. 2022Flow Annealed Importance Sampling Bootstrapcited by 124
  6. 2024Iterated Denoising Energy Matching for Sampling from Boltzmann Densitiescited by 113
  7. 2016R'enyi Divergence Variational Inferencecited by 104
  8. 2022GFlowNets and variational inferencecited by 103
  9. 2023Improved sampling via learned diffusionscited by 103
  10. 2023A theory of continuous generative flow networkscited by 98
  11. 2021Annealed Flow Transport Monte Carlocited by 90
  12. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  13. 2022Continual Repeated Annealed Flow Transport Monte Carlocited by 66
  14. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  15. 2021Bayesian Learning via Neural Schr"odinger-F"ollmer Flowscited by 56
  16. 2022Score-Based Diffusion meets Annealed Importance Samplingcited by 51
  17. 2021MCMC Variational Inference via Uncorrected Hamiltonian Annealingcited by 39
  18. 2023SE(3) Equivariant Augmented Coupling Flowscited by 37
  19. 2021Monte Carlo Variational Auto-Encoderscited by 36
  20. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  21. 2020f-Divergence Variational Inferencecited by 28
  22. 2022Langevin Diffusion Variational Inferencecited by 26
  23. 2022A Unified Perspective on Natural Gradient Variational Inference with Gaussian Mixture Modelscited by 22
  24. 2021Surrogate Likelihoods for Variational Annealed Importance Samplingcited by 14
  25. 2019Stochastic Variational Inference via Upper Boundcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 32; 2026: 13.

55 papers.

Sort:

By area:

  1. 2024NETS: A Non-Equilibrium Transport Samplercited by 67
  2. 2024Sequential Controlled Langevin Diffusionscited by 47
  3. 2025Underdamped Diffusion Bridges with Applications to Samplingcited by 28
  4. 2025Progressive Inference-Time Annealing of Diffusion Models for Sampling from Boltzmann Densitiescited by 26
  5. 2025No Trick, No Treat: Pursuits and Challenges Towards Simulation-free Training of Neural Samplerscited by 25
  6. 2024Adaptive teachers for amortized samplerscited by 23
  7. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 22
  8. 2025Adjoint Schr"odinger Bridge Samplercited by 20
  9. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 17
  10. 2025Proximal Diffusion Neural Samplercited by 16
  11. 2025FEAT: Free energy Estimators with Adaptive Transportcited by 15
  12. 2024Variational Distillation of Diffusion Policies into Mixture of Expertscited by 14
38 more
  1. 2024A theoretical perspective on mode collapse in variational inferencecited by 14
  2. 2025Temperature-Annealed Boltzmann Generatorscited by 14
  3. 2025Progressive Tempering Sampler with Diffusioncited by 13
  4. 2024Harmonic Path Integral Diffusioncited by 11
  5. 2025On scalable and efficient training of diffusion samplerscited by 10
  6. 2024Learned Reference-based Diffusion Sampling for multi-modal distributionscited by 9
  7. 2025Improving the evaluation of samplers on multi-modal targetscited by 9
  8. 2025CREPE: Controlling Diffusion with Replica Exchangecited by 9
  9. 2025Non-equilibrium Annealed Adjoint Samplercited by 8
  10. 2025Learning Boltzmann Generators via Constrained Mass Transportcited by 8
  11. 2024Importance Corrected Neural JKO Samplingcited by 7
  12. 2025Rethinking Losses for Diffusion Bridge Samplerscited by 7
  13. 2026Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matchingcited by 7
  14. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  15. 2025Variational Inference with Mixtures of Isotropic Gaussianscited by 6
  16. 2024Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VIcited by 5
  17. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
  18. 2025In-Context Parametric Inference: Point or Distribution Estimators?cited by 4
  19. 2025End-To-End Learning of Gaussian Mixture Priors for Diffusion Samplercited by 4
  20. 2025Model-Informed Flows for Bayesian Inferencecited by 3
  21. 2025SESaMo: Symmetry-Enforcing Stochastic Modulation for Normalizing Flowscited by 2
  22. 2026Efficient Training of Boltzmann Generators Using Off-Policy Log-Dispersion Regularizationcited by 2
  23. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  24. 2026Annealing in variational inference mitigates mode collapse: A theoretical study on Gaussian mixturescited by 2
  25. 2024An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Controlcited by 1
  26. 2025Enhancing Gradient-based Discrete Sampling via Parallel Temperingcited by 1
  27. 2025Sampling by averaging: A multiscale approach to score estimationcited by 1
  28. 2025Continuously Tempered Diffusion Samplerscited by 1
  29. 2025Rotated Mean-Field Variational Inference and Iterative Gaussianizationcited by 1
  30. 2025One-Step Diffusion Samplers via Self-Distillation and Deterministic Flowcited by 1
  31. 2025Adaptive Path Integral Diffusion: AdaPIDcited by 1
  32. 2026How to Approximate Inference with Subtractive Mixture Modelscited by 1
  33. 2026Autoregressive Boltzmann Generatorscited by 1
  34. 2025Exploring Non-Convex Discrete Energy Landscapes: An Efficient Langevin-Like Sampler with Replica Exchangecited by 0
  35. 2025Value Gradient Sampler: Learning Invariant Value Functions for Equivariant Diffusion Samplingcited by 0
  36. 2025Mode Collapse of Mean-Field Variational Inferencecited by 0
  37. 2026Total Variation Guarantees for Sampling with Stochastic Localizationcited by 0
  38. 2026Variational Autoregressive Networks with probability priorscited by 0

50 most cited of 55 All 55 (JSON)