# Moment Matching Denoising Gibbs Sampling

arXiv: https://arxiv.org/abs/2305.11650 · HTML: https://mlsanity.com/paper/2305.11650/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 52 references that are in the map.
- First posted on arXiv 19 May 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [A Theory of Generative ConvNet](https://mlsanity.com/paper/1602.03264/) (2016)
- [Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model](https://mlsanity.com/paper/1904.09770/) (2019)
- [Improved Contrastive Divergence Training of Energy Based Models](https://mlsanity.com/paper/2012.01316/) (2020)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [Adversarial score matching and improved sampling for image generation](https://mlsanity.com/paper/2009.05475/) (2020)
- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [Deep Directed Generative Models with Energy-Based Probability Estimation](https://mlsanity.com/paper/1606.03439/) (2016)
- [Deep Energy Estimator Networks](https://mlsanity.com/paper/1805.08306/) (2018)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [Learning Energy-Based Models as Generative ConvNets via Multi-grid Modeling and Sampling](https://mlsanity.com/paper/1709.08868/) (2017)
- [Adversarial Contrastive Estimation](https://mlsanity.com/paper/1805.03642/) (2018)
- [Spread Divergence](https://mlsanity.com/paper/1811.08968/) (2018)
- [On approximating nabla f with neural networks](https://mlsanity.com/paper/1910.12744/) (2019)
- [Variational f-divergence Minimization](https://mlsanity.com/paper/1907.11891/) (2019)
- [Towards Healing the Blindness of Score Matching](https://mlsanity.com/paper/2209.07396/) (2022)
- [Generative Adversarial Networks as Variational Training of Energy Based Models](https://mlsanity.com/paper/1611.01799/) (2016)
- [Generalization Gap in Amortized Inference](https://mlsanity.com/paper/2205.11640/) (2022)
- [A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models](https://mlsanity.com/paper/2002.07501/) (2020)

## Built on by (most cited first, 7 of 7)

- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)
- [Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching](https://mlsanity.com/paper/2406.10808/) (2024)
- [Inference-Time Scaling of Diffusion Language Models via Trajectory Refinement](https://mlsanity.com/paper/2507.08390/) (2025)
- [Outlier-robust Diffusion Posterior Sampling for Bayesian Inverse Problems](https://mlsanity.com/paper/2602.02045/) (2026)
- [Closed-Form Beta Distribution Estimation from Sparse Statistics with Random Forest Implicit Regularization](https://mlsanity.com/paper/2507.23767/) (2025)
- [Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences](https://mlsanity.com/paper/2605.26756/) (2026)

## Abstract (arXiv)

Energy-Based Models (EBMs) offer a versatile framework for modeling complex data distributions. However, training and sampling from EBMs continue to pose significant challenges. The widely-used Denoising Score Matching (DSM) method for scalable EBM training suffers from inconsistency issues, causing the energy model to learn a `noisy' data distribution. In this work, we propose an efficient sampling framework: (pseudo)-Gibbs sampling with moment matching, which enables effective sampling from the underlying clean model when given a `noisy' model that has been well-trained via DSM. We explore the benefits of our approach compared to related methods and demonstrate how to scale the method to high-dimensional datasets.