mlsanityData to 14 Oct 2026

Local Curvature Smoothing with Stein's Identity for Efficient Score Matching

arXiv 2412.03962 ↗First posted 5 Dec 2024cs.LG cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The training of score-based diffusion models (SDMs) is based on score matching. The challenge of score matching is that it includes a computationally expensive Jacobian trace. While several methods have been proposed to avoid this computation, each has drawbacks, such as instability during training and approximating the learning as learning a denoising vector field rather than a true score. We propose a novel score matching variant, local curvature smoothing with Stein's identity (LCSS). The LCSS bypasses the Jacobian trace by applying Stein's identity, enabling regularization effectiveness and efficient computation. We show that LCSS surpasses existing methods in sample generation performance and matches the performance of denoising score matching, widely adopted by most SDMs, in evaluations such as FID, Inception score, and bits per dimension. Furthermore, we show that LCSS enables realistic image generation even at a high resolution of 1024 × 1024.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 28 of its 40 references that are in the map. First posted on arXiv 5 Dec 2024.

Builds on

28 of 40 references.

Closest ancestors · 8

Same direction, newest first

  1. 2022FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation9 Oct
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  3. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Models27 May
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Probabilistic Models19 Jun
  8. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 20

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  8. 2015Variational Inference with Normalizing Flowscited by 2,170
  9. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  10. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  11. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  12. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
8 more
  1. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  2. 2016A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluationcited by 236
  3. 2015Measuring Sample Quality with Stein's Methodcited by 94
  4. 2013RNADE: The real-valued neural autoregressive density-estimatorcited by 66
  5. 2017Gradient Estimators for Implicit Modelscited by 58
  6. 2020Efficient Learning of Generative Models via Finite-Difference Score Matchingcited by 49
  7. 2018A Spectral Approach to Gradient Estimation for Implicit Distributionscited by 45
  8. 2020AR-DAE: Towards Unbiased Neural Entropy Gradient Estimationcited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation
  2. 2025Optimizing Input of Denoising Score Matching is Biased Towards Higher Score Norm
  3. 2024From optimal score matching to optimal sampling
  4. 2025A Connection Between Score Matching and Local Intrinsic Dimension
  5. 2026Sobolev Regularized Score Difference Estimation in Diffusion Models
  6. 2024What's the score? Automated Denoising Score Matching for Nonlinear Diffusions
  7. 2022From Denoising Diffusions to Denoising Markov Models
  8. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models