mlsanityData to 7 Oct 2026

Improved off-policy training of diffusion samplers

arXiv 2402.05098 ↗First posted 7 Feb 2024cs.LG stat.MLcited by 47

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 62 of its 93 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

62 of 93 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Denoising Diffusion Samplers27 Feb
  2. 2022An optimal control perspective on diffusion-based generative modeling2 Nov
  3. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  6. 2021Diffusion Normalizing Flow14 Oct
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 52

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2016Density estimation using Real NVPcited by 2,029
  5. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  6. 2015Importance Weighted Autoencoderscited by 454
  7. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  8. 2013Black Box Variational Inferencecited by 317
  9. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  10. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  11. 2021GFlowNet Foundationscited by 286
  12. 2022Torsional Diffusion for Molecular Conformer Generationcited by 262
38 more
  1. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  2. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  3. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  4. 2022Biological Sequence Design with GFlowNetscited by 172
  5. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  6. 2020Stochastic Normalizing Flowscited by 156
  7. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  8. 2019Flow-based generative models for Markov chain Monte Carlo in lattice field theorycited by 147
  9. 2020BayesFlow: Learning complex stochastic models with invertible neural networkscited by 139
  10. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  11. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 121
  12. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  13. 2022GFlowNets and variational inferencecited by 103
  14. 2023Improved sampling via learned diffusionscited by 103
  15. 2023A theory of continuous generative flow networkscited by 98
  16. 2019Theoretical guarantees for sampling and inference in generative models with latent diffusionscited by 97
  17. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
  18. 2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 80
  19. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  20. 2019NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transportcited by 73
  21. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  22. 2023Robust Scheduling with GFlowNetscited by 67
  23. 2023Generative Flow Networks as Entropy-Regularized RLcited by 60
  24. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  25. 2022A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inferencecited by 50
  26. 2023Sample-efficient Multi-objective Molecular Optimization with GFlowNetscited by 48
  27. 2023Local Search GFlowNetscited by 45
  28. 2023Learning Interpolations between Boltzmann Densitiescited by 42
  29. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  30. 2022A Variational Perspective on Generative Flow Networkscited by 39
  31. 2022Stochastic Optimal Control for Collective Variable Free Sampling of Molecular Transition Pathscited by 36
  32. 2023Thompson sampling for improved exploration in GFlowNetscited by 31
  33. 2023DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNetscited by 27
  34. 2020i-flow: High-dimensional Integration and Sampling with Normalizing Flowscited by 26
  35. 2023Learning to Scale Logits for Temperature-Conditional GFlowNetscited by 26
  36. 2023Learning Energy Decompositions for Partial Inference of GFlowNetscited by 24
  37. 2022Posterior samples of source galaxies in strong gravitational lenses with score-based priorscited by 20
  38. 2021Amortized Variational Inference for Simple Hierarchical Modelscited by 19

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 26; 2026: 8.

47 papers.

Sort:

By area:

  1. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learningcited by 70
  2. 2024NETS: A Non-Equilibrium Transport Samplercited by 67
  3. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
  4. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 29
  5. 2025Maximum Entropy Reinforcement Learning with Diffusion Policycited by 29
  6. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
  7. 2024Adaptive teachers for amortized samplerscited by 23
  8. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 22
  9. 2024Training Neural Samplers with Reverse Diffusive KL Divergencecited by 21
  10. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 17
  11. 2025Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative modelscited by 16
  12. 2024BNEM: A Boltzmann Sampler Based on Bootstrapped Noised Energy Matchingcited by 14
35 more
  1. 2025Temperature-Annealed Boltzmann Generatorscited by 14
  2. 2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
  3. 2025Progressive Tempering Sampler with Diffusioncited by 13
  4. 2024Can a Bayesian Oracle Prevent Harm from an Agent?cited by 12
  5. 2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimizationcited by 12
  6. 2024Harmonic Path Integral Diffusioncited by 11
  7. 2025On scalable and efficient training of diffusion samplerscited by 10
  8. 2024Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flowscited by 9
  9. 2025Rethinking Losses for Diffusion Bridge Samplerscited by 7
  10. 2026Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matchingcited by 7
  11. 2024MetaGFN: Exploring Distant Modes with Adapted Metadynamics for Continuous GFlowNetscited by 6
  12. 2025Iterative Importance Fine-tuning of Diffusion Modelscited by 6
  13. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  14. 2025Data-to-Energy Stochastic Dynamicscited by 5
  15. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
  16. 2025Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Functioncited by 5
  17. 2024Amortized Inference of Causal Models via Conditional Fixed-Point Iterationscited by 4
  18. 2025In-Context Parametric Inference: Point or Distribution Estimators?cited by 4
  19. 2025Loss-Guided Auxiliary Agents for Overcoming Mode Collapse in GFlowNetscited by 4
  20. 2025Posterior Inference in Latent Space for Scalable Constrained Black-box Optimizationcited by 4
  21. 2025Control Variate Score Matching for Diffusion Modelscited by 4
  22. 2024Action abstractions for amortized samplingcited by 3
  23. 2025Energy-based generator matching: A neural sampler for general state spacecited by 3
  24. 2025Ergodic Generative Flowscited by 2
  25. 2025Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coveragecited by 2
  26. 2026Efficient Training of Boltzmann Generators Using Off-Policy Log-Dispersion Regularizationcited by 2
  27. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  28. 2026Constrained Diffusion Models with Primal-Dual Inferencecited by 2
  29. 2025Adaptive Path Integral Diffusion: AdaPIDcited by 1
  30. 2025Solving Bayesian inverse problems with diffusion priors and off-policy RLcited by 0
  31. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Trainingcited by 0
  32. 2026Signal from Structure: Exploiting Submodular Upper Bounds in Generative Flow Networkscited by 0
  33. 2026A Unified Framework for Data-Free One-Step Sampling via Wasserstein Gradient Flowscited by 0
  34. 2026Proximal Policy Optimization for Amortized Discrete Samplingcited by 0
  35. 2026Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
  2. 2025Single-Step Consistent Diffusion Samplers
  3. 2025Diffusion Models: A Mathematical Introduction
  4. 2025Pretrained Diffusion Models Are Inherently Skipped-Step Samplers
  5. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers
  6. 2026Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning
  7. 2024Amortizing intractable inference in diffusion models for vision, language, and control
  8. 2025Understanding Sampler Stochasticity in Training Diffusion Models for RLHF