# On tuning consistent annealed sampling for denoising score matching

arXiv: https://arxiv.org/abs/2104.03725 · HTML: https://mlsanity.com/paper/2104.03725/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 9 of its 11 references that are in the map.
- First posted on arXiv 8 Apr 2021.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 9 of 9)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Adversarial score matching and improved sampling for image generation](https://mlsanity.com/paper/2009.05475/) (2020)
- [SESQA: semi-supervised learning for speech quality assessment](https://mlsanity.com/paper/2010.00368/) (2020)

## Built on by (most cited first, 4 of 4)

- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [Full-band General Audio Synthesis with Score-based Diffusion](https://mlsanity.com/paper/2210.14661/) (2022)
- [Distribution Preserving Source Separation With Time Frequency Predictive Models](https://mlsanity.com/paper/2303.05896/) (2023)
- [Audio Decoding by Inverse Problem Solving](https://mlsanity.com/paper/2409.07858/) (2024)

## Abstract (arXiv)

Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized series of noise levels and a predefined scheme. In this note, we first overview three common sampling schemes for models trained with denoising score matching. Next, we focus on one of them, consistent annealed sampling, and study its hyper-parameter boundaries. We then highlight a possible formulation of such hyper-parameter that explicitly considers those boundaries and facilitates tuning when using few or a variable number of steps. Finally, we highlight some connections of the formulation with other sampling schemes.