mlsanityData to 7 Oct 2026

Generative Modelling with High-Order Langevin Dynamics

arXiv 2404.12814 ↗First posted 19 Apr 2024cs.LG cs.AI cs.CVcited by 5

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion generative modelling (DGM) based on stochastic differential equations (SDEs) with score matching has achieved unprecedented results in data generation. In this paper, we propose a novel fast high-quality generative modelling method based on high-order Langevin dynamics (HOLD) with score matching. This motive is proved by third-order Langevin dynamics. By augmenting the previous SDEs, e.g. variance exploding or variance preserving SDEs for single-data variable processes, HOLD can simultaneously model position, velocity, and acceleration, thereby improving the quality and speed of the data generation at the same time. HOLD is composed of one Ornstein-Uhlenbeck process and two Hamiltonians, which reduce the mixing time by two orders of magnitude. Empirical experiments for unconditional image generation on the public data set CIFAR-10 and CelebA-HQ show that the effect is significant in both Frechet inception distance (FID) and negative log-likelihood, and achieves the state-of-the-art FID of 1.85 on CIFAR-10.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 34 of its 53 references that are in the map. First posted on arXiv 19 Apr 2024.

Builds on

34 of 53 references.

Closest ancestors · 16

Same direction, newest first

  1. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  2. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  7. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  8. 2021Score-based Generative Modeling in Latent Space10 Jun
  9. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  10. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  11. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  12. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
4 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Implicit Models6 Oct
  3. 2020Denoising Diffusion Probabilistic Models19 Jun
  4. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  11. 2019Residual Flows for Invertible Generative Modelingcited by 256
  12. 2020Learning Energy-Based Models by Diffusion Recovery Likelihoodcited by 122
6 more
  1. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  2. 2021Deep Generative Learning via Schr"odinger Bridgecited by 103
  3. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Schemecited by 99
  4. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  5. 2020Dual Contradistinctive Generative Autoencodercited by 58
  6. 2019High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithmcited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2025Underdamped Diffusion Bridges with Applications to Samplingcited by 28
  2. 2024Critically Damped Third-Order Langevin Dynamicscited by 3
  3. 2025Critically-Damped Higher-Order Langevin Dynamics for Generative Modelingcited by 3
  4. 2025Defending Diffusion Models Against Membership Inference Attacks via Higher-Order Langevin Dynamicscited by 1
  5. 2026Reducing Diffusion Model Memorization with Higher Order Langevin Dynamicscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Score-based constrained generative modeling via Langevin diffusions with boundary conditions
  2. 2024On the Hardness of Sampling from Mixture Distributions via Langevin Dynamics
  3. 2025Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling
  4. 2023A Complete Recipe for Diffusion Generative Models
  5. 2026Training-Free Generative Sampling via Moment-Matched Score Smoothing
  6. 2025A Malliavin calculus approach to score functions in diffusion generative models
  7. 2022Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling
  8. 2022A Flexible Diffusion Model