mlsanityData to 14 Oct 2026

Score-based Generative Models with Adaptive Momentum

arXiv 2405.13726 ↗First posted 22 May 2024cs.LGcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based generative models have demonstrated significant practical success in data-generating tasks. The models establish a diffusion process that perturbs the ground truth data to Gaussian noise and then learn the reverse process to transform noise into data. However, existing denoising methods such as Langevin dynamic and numerical stochastic differential equation solvers enjoy randomness but generate data slowly with a large number of score function evaluations, and the ordinary differential equation solvers enjoy faster sampling speed but no randomness may influence the sample quality. To this end, motivated by the Stochastic Gradient Descent (SGD) optimization methods and the high connection between the model sampling process with the SGD, we propose adaptive momentum sampling to accelerate the transforming process without introducing additional hyperparameters. Theoretically, we proved our method promises convergence under given conditions. In addition, we empirically show that our sampler can produce more faithful images/graphs in small sampling steps with 2 to 5 times speed up and obtain competitive scores compared to the baselines on image and graph generation tasks.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 28 of its 42 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

28 of 42 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Boosting Diffusion Models with Moving Average Sampling in Frequency Domain26 Mar
  2. 2023Boosting Diffusion Models with an Adaptive Momentum Sampler23 Aug
  3. 2023Diffusion Sampling with Momentum for Mitigating Divergence Artifacts20 Jul
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  7. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun
  11. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 17

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  5. 2022Video Diffusion Modelscited by 2,305
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  7. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 520
  8. 2018GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Modelscited by 413
  9. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  10. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 283
  11. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 270
  12. 2019Efficient Graph Generation with Graph Recurrent Attention Networkscited by 234
5 more
  1. 2018Learning Deep Generative Models of Graphscited by 229
  2. 2021GraphDF: A Discrete Flow Model for Molecular Graph Generationcited by 180
  3. 2020Adversarial score matching and improved sampling for image generationcited by 96
  4. 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 88
  5. 2021Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimizationcited by 8

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025RODS: Robust Optimization Inspired Diffusion Sampling for Detecting and Reducing Hallucination in Generative Modelscited by 3
  2. 2025PID-controlled Langevin Dynamics for Faster Sampling of Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
  2. 2023Preconditioned Score-based Generative Models
  3. 2022Convergence of score-based generative modeling for general data distributions
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
  5. 2026A Stochastic Analysis Perspective on Diffusion Models
  6. 2022Accelerating Score-based Generative Models for High-Resolution Image Synthesis
  7. 2022Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling
  8. 2024Discrete vs. Continuous Trade-offs for Generative Models