mlsanityData to 14 Oct 2026

Moment Matching Denoising Gibbs Sampling

arXiv 2305.11650 ↗First posted 19 May 2023stat.ML cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Energy-Based Models (EBMs) offer a versatile framework for modeling complex data distributions. However, training and sampling from EBMs continue to pose significant challenges. The widely-used Denoising Score Matching (DSM) method for scalable EBM training suffers from inconsistency issues, causing the energy model to learn a `noisy' data distribution. In this work, we propose an efficient sampling framework: (pseudo)-Gibbs sampling with moment matching, which enables effective sampling from the underlying clean model when given a `noisy' model that has been well-trained via DSM. We explore the benefits of our approach compared to related methods and demonstrate how to scale the method to high-dimensional datasets.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 28 of its 52 references that are in the map. First posted on arXiv 19 May 2023.

Builds on

28 of 52 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  8. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 334
  9. 2021How to Train Your Energy-Based Modelscited by 274
  10. 2013Expectation Propagation for approximate Bayesian inferencecited by 174
  11. 2016A Theory of Generative ConvNetcited by 174
  12. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 148
16 more
  1. 2020Improved Contrastive Divergence Training of Energy Based Modelscited by 130
  2. 2013Generalized Denoising Auto-Encoders as Generative Modelscited by 113
  3. 2020Adversarial score matching and improved sampling for image generationcited by 96
  4. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 64
  5. 2016Deep Directed Generative Models with Energy-Based Probability Estimationcited by 34
  6. 2018Deep Energy Estimator Networkscited by 34
  7. 2020Blindness of score-based methods to isolated components and mixing proportionscited by 34
  8. 2017Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Samplingcited by 32
  9. 2018Adversarial Contrastive Estimationcited by 23
  10. 2018Spread Divergencecited by 23
  11. 2019On approximating nabla f with neural networkscited by 18
  12. 2019Variational f-divergence Minimizationcited by 16
  13. 2022Towards Healing the Blindness of Score Matchingcited by 13
  14. 2016Generative Adversarial Networks as Variational Training of Energy Based Modelscited by 10
  15. 2022Generalization Gap in Amortized Inferencecited by 10
  16. 2020A Wasserstein Minimum Velocity Approach to Learning Unnormalized Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 2.

7 papers.

  1. 2024Diffusive Gibbs Samplingcited by 21
  2. 2024Training Neural Samplers with Reverse Diffusive KL Divergencecited by 21
  3. 2024Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matchingcited by 13
  4. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinementcited by 7
  5. 2026Outlier-robust Diffusion Posterior Sampling for Bayesian Inverse Problemscited by 1
  6. 2025Closed-Form Beta Distribution Estimation from Sparse Statistics with Random Forest Implicit Regularizationcited by 0
  7. 2026Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differencescited by 0