# Selective Guidance: Are All the Denoising Steps of Guided Diffusion Important?

arXiv: https://arxiv.org/abs/2305.09847 · HTML: https://mlsanity.com/paper/2305.09847/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 4 of its 6 references that are in the map.
- First posted on arXiv 16 May 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 4 of 4)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)

## Built on by (most cited first, 5 of 5)

- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Plug-and-Play Diffusion Distillation](https://mlsanity.com/paper/2406.01954/) (2024)
- [SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration](https://mlsanity.com/paper/2409.02231/) (2024)
- [LoRA-Enhanced Distillation on Guided Diffusion Models](https://mlsanity.com/paper/2312.06899/) (2023)
- [Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration](https://mlsanity.com/paper/2511.04117/) (2025)

## Abstract (arXiv)

This study examines the impact of optimizing the Stable Diffusion (SD) guided inference pipeline. We propose optimizing certain denoising steps by limiting the noise computation to conditional noise and eliminating unconditional noise computation, thereby reducing the complexity of the target iterations by 50%. Additionally, we demonstrate that later iterations of the SD are less sensitive to optimization, making them ideal candidates for applying the suggested optimization. Our experiments show that optimizing the last 20% of the denoising loop iterations results in an 8.2% reduction in inference time with almost no perceivable changes to the human eye. Furthermore, we found that by extending the optimization to 50% of the last iterations, we can reduce inference time by approximately 20.3%, while still generating visually pleasing images.