mlsanityData to 7 Oct 2026

Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions

arXiv 2409.20547 ↗First posted 30 Sep 2024stat.ML cs.LG stat.COcited by 11

Not on the map. Why?

Abstract (arXiv)

Sampling from high-dimensional, multi-modal distributions remains a fundamental challenge across domains such as statistical Bayesian inference and physics-based machine learning. In this paper, we propose Annealing Flow (AF), a method built on Continuous Normalizing Flow (CNF) for sampling from high-dimensional and multi-modal distributions. AF is trained with a dynamic Optimal Transport (OT) objective incorporating Wasserstein regularization, and guided by annealing procedures, facilitating effective exploration of modes in high-dimensional spaces. Compared to recent NF methods, AF greatly improves training efficiency and stability, with minimal reliance on MC assistance. We demonstrate the superior performance of AF compared to state-of-the-art methods through experiments on various challenging distributions and real-world datasets, particularly in high-dimensional and multi-modal settings. We also highlight AF potential for sampling the least favorable distributions.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 43 of its 85 references that are in the map. First posted on arXiv 30 Sep 2024.

Builds on

43 of 85 references.

References · 43

  1. 2015Variational Inference with Normalizing Flowscited by 2,161
  2. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  3. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  5. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  6. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
  7. 2020Stochastic Normalizing Flowscited by 156
  8. 2017Stein Variational Gradient Descent as Gradient Flowcited by 149
  9. 2023Parallel Sampling of Diffusion Modelscited by 112
  10. 2020Telescoping Density-Ratio Estimationcited by 108
  11. 2022Accelerating Bayesian Optimization for Biological Sequence Design with Denoising Autoencoderscited by 99
  12. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
31 more
  1. 2021Annealed Flow Transport Monte Carlocited by 90
  2. 2022Continual Repeated Annealed Flow Transport Monte Carlocited by 66
  3. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 59
  4. 2020Scaling Hamiltonian Monte Carlo Inference for Bayesian Neural Networks with Symmetric Splittingcited by 54
  5. 2017Stochastic Particle Gradient Descent for Infinite Ensemblescited by 52
  6. 2018A Stein variational Newton methodcited by 49
  7. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Modelscited by 46
  8. 2024Learning general Gaussian mixtures with efficient score matchingcited by 43
  9. 2022Normalizing flow neural networks by JKO schemecited by 39
  10. 2021Differentiable Annealed Importance Sampling and the Perils of Gradient Noisecited by 36
  11. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  12. 2021Stochastic Normalizing Flows for Inverse Problems: a Markov Chains Viewpointcited by 32
  13. 2024Sampling in Unit Time with Kernel Fisher-Rao Flowcited by 27
  14. 2024Liouville Flow Importance Samplercited by 25
  15. 2015Provable Bayesian Inference via Particle Mirror Descentcited by 23
  16. 2022Sampling with Mollified Interaction Energy Descentcited by 17
  17. 2021Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methodscited by 15
  18. 2018Langevin-gradient parallel tempering for Bayesian neural learningcited by 14
  19. 2023Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyondcited by 13
  20. 2023Flow-based Distributionally Robust Optimizationcited by 13
  21. 2020A Neural Network MCMC sampler that maximizes Proposal Entropycited by 12
  22. 2024Posterior Sampling with Denoising Oracles via Tilted Transportcited by 12
  23. 2023Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initializationcited by 10
  24. 2024Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flowscited by 9
  25. 2023Computing high-dimensional optimal transport by flow neural networkscited by 8
  26. 2019Scalable Metropolis-Hastings for Exact Bayesian Inference with Large Datasetscited by 4
  27. 2022Improving control based importance sampling strategies for metastable diffusions via adapted metadynamicscited by 4
  28. 2023Efficient Multimodal Sampling via Tempered Distribution Flowcited by 3
  29. 2023Bayesian-Boosted MetaLoc: Efficient Training and Guaranteed Generalization for Indoor Localizationcited by 3
  30. 2020Bayesian model averaging for analysis of lattice field theory resultscited by 2
  31. 2024Ai-Sampler: Adversarial Learning of Markov kernels with involutive mapscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 4.

11 papers.

  1. 2025Density Ratio Estimation with Conditional Probability Pathscited by 13
  2. 2024Nonlinear Assimilation via Score-based Sequential Langevin Samplingcited by 7
  3. 2025Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Samplingcited by 6
  4. 2025DoFlow: Flow-based Generative Models for Interventional and Counterfactual Forecasting on Time Seriescited by 3
  5. 2025FlowVAT: Normalizing Flow Variational Inference with Affine-Invariant Temperingcited by 2
  6. 2025Flow-based Generative Modeling of Potential Outcomes and Counterfactualscited by 1
  7. 2025Iterative Sampling Methods for Sinkhorn Distributionally Robust Optimizationcited by 1
  8. 2026CoreFlow: Low-Rank Matrix Generative Modelscited by 1
  9. 2026Sampling via Stochastic Interpolants by Langevin-based Velocity and Initialization Estimation in Flow ODEscited by 0
  10. 2026Generative Nested Sampling of Atomistic Thermodynamic Landscapescited by 0
  11. 2026GazeDiT: Gaze-Accurate Diffusion Image Generation for Eye Tracking via Spatial Conditioningcited by 0