mlsanityData to 7 Oct 2026

From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training

arXiv 2501.06148 ↗First posted 10 Jan 2025cs.LG stat.MLcited by 17

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

We study the problem of training neural stochastic differential equations, or diffusion models, to sample from a Boltzmann distribution without access to target samples. Existing methods for training such models enforce time-reversal of the generative and noising processes, using either differentiable simulation or off-policy reinforcement learning (RL). We prove equivalences between families of objectives in the limit of infinitesimal discretization steps, linking entropic RL methods (GFlowNets) with continuous-time objects (partial differential equations and path space measures). We further show that an appropriate choice of coarse time discretization during training allows greatly improved sample efficiency and the use of time-local objectives, achieving competitive performance on standard sampling benchmarks with reduced computational cost.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 60 of its 96 references that are in the map. First posted on arXiv 10 Jan 2025.

Builds on

60 of 96 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control13 Sep

Other references · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  7. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  8. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  9. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
  10. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  11. 2013A survey of the Schr"odinger problem and some of its connections with optimal transportcited by 286
  12. 2021GFlowNet Foundationscited by 286
38 more
  1. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  2. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  3. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  4. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  5. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  6. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  7. 2021Neural SDEs as Infinite-Dimensional GANscited by 159
  8. 2020Stochastic Normalizing Flowscited by 156
  9. 2019Flow-based generative models for Markov chain Monte Carlo in lattice field theorycited by 147
  10. 2022An optimal control perspective on diffusion-based generative modelingcited by 145
  11. 2023Denoising Diffusion Samplerscited by 144
  12. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  13. 2022Flow Annealed Importance Sampling Bootstrapcited by 124
  14. 2024Iterated Denoising Energy Matching for Sampling from Boltzmann Densitiescited by 113
  15. 2022GFlowNets and variational inferencecited by 103
  16. 2023Improved sampling via learned diffusionscited by 103
  17. 2023A theory of continuous generative flow networkscited by 98
  18. 2019A Machine Learning Framework for Solving High-Dimensional Mean Field Game and Mean Field Control Problemscited by 95
  19. 2021Annealed Flow Transport Monte Carlocited by 90
  20. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
  21. 2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 80
  22. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  23. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  24. 2021Efficient and Accurate Gradients for Neural SDEscited by 68
  25. 2022Continual Repeated Annealed Flow Transport Monte Carlocited by 66
  26. 2024Particle Denoising Diffusion Samplercited by 63
  27. 2023Generative Flow Networks as Entropy-Regularized RLcited by 60
  28. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  29. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 55
  30. 2022Score-Based Diffusion meets Annealed Importance Samplingcited by 51
  31. 2024Improved off-policy training of diffusion samplerscited by 47
  32. 2024DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised h-transformcited by 44
  33. 2023Learning Interpolations between Boltzmann Densitiescited by 42
  34. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 40
  35. 2023Bespoke Solvers for Generative Flow Modelscited by 36
  36. 2020An invitation to sequential Monte Carlo samplerscited by 33
  37. 2021Generalized Normalizing Flows via Markov Chainscited by 23
  38. 2021Interpolating between BSDEs and PINNs: deep learning for elliptic and parabolic boundary value problemscited by 21

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 5.

17 papers.

Sort:

By area:

  1. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 22
  2. 2025Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative modelscited by 16
  3. 2025RNE: plug-and-play diffusion inference-time control and energy-based trainingcited by 16
  4. 2025Accelerated Parallel Tempering via Neural Transportscited by 13
  5. 2025On scalable and efficient training of diffusion samplerscited by 10
  6. 2025CREPE: Controlling Diffusion with Replica Exchangecited by 9
  7. 2026Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matchingcited by 7
  8. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  9. 2025Data-to-Energy Stochastic Dynamicscited by 5
  10. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
  11. 2025Energy-based generator matching: A neural sampler for general state spacecited by 3
  12. 2026f-Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Datacited by 3
5 more
  1. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  2. 2025One-Step Diffusion Samplers via Self-Distillation and Deterministic Flowcited by 1
  3. 2025Assessing generative modeling approaches for free energy estimates in condensed mattercited by 1
  4. 2026Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernelscited by 0
  5. 2026Tensor Train Diffusion: Leveraging Low-Rank Structures for High-Dimensional Score-Based Samplingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ART for Diffusion Sampling: A Reinforcement Learning Approach to Timestep Schedule
  2. 2026ART for Diffusion Sampling: Continuous-Time Control and Actor-Critic Learning
  3. 2026Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies
  4. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
  5. 2024Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning
  6. 2025Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
  7. 2026Diffusion Controller: Framework, Algorithms and Parameterization
  8. 2026Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models