mlsanityData to 14 Oct 2026

Score Normalization for a Faster Diffusion Exponential Integrator Sampler

arXiv 2311.00157 ↗First posted 31 Oct 2023cs.LG cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Recently, Zhang et al. have proposed the Diffusion Exponential Integrator Sampler (DEIS) for fast generation of samples from Diffusion Models. It leverages the semi-linear nature of the probability flow ordinary differential equation (ODE) in order to greatly reduce integration error and improve generation quality at low numbers of function evaluations (NFEs). Key to this approach is the score function reparameterisation, which reduces the integration error incurred from using a fixed score function estimate over each integration step. The original authors use the default parameterisation used by models trained for noise prediction -- multiply the score by the standard deviation of the conditional forward noising distribution. We find that although the mean absolute value of this score parameterisation is close to constant for a large portion of the reverse sampling process, it changes rapidly at the end of sampling. As a simple fix, we propose to instead reparameterise the score (at inference) by dividing it by the average absolute value of previous score estimates at that time step collected from offline high NFE generations. We find that our score normalisation (DEIS-SN) consistently improves FID compared to vanilla DEIS, showing an improvement at 10 NFEs from 6.44 to 5.57 on CIFAR-10 and from 5.9 to 4.95 on LSUN-Church 64x64. Our code is available at https://github.com/mtkresearch/Diffusion-DEIS-SN

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 25 references that are in the map. First posted on arXiv 31 Oct 2023.

Builds on

22 of 25 references.

References · 22

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  12. 2021Variational Diffusion Modelscited by 1,153
10 more
  1. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  2. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 599
  3. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 532
  4. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  5. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 233
  6. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 182
  7. 2022Fast Sampling of Diffusion Models via Operator Learningcited by 175
  8. 2022gDDIM: Generalized denoising diffusion implicit modelscited by 125
  9. 2022Blurring Diffusion Modelscited by 91
  10. 2023Image generation with shortest path diffusioncited by 8

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024DiscDiff: Latent Diffusion Model for DNA Sequence Generationcited by 12
  2. 2024Towards Understanding and Quantifying Uncertainty for Text-to-Image Generationcited by 11

Similar, not cited

Close in wording, no citation either way

  1. 2024Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
  2. 2023Denoising Diffusion Samplers
  3. 2026Diffusion Path Samplers via Sequential Monte Carlo
  4. 2023Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling
  5. 2024Provable Acceleration for Diffusion Models under Minimal Assumptions
  6. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably
  7. 2025Faster Diffusion Models via Higher-Order Approximation
  8. 2025Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models