# Navigating with Annealing Guidance Scale in Diffusion Space

arXiv: https://arxiv.org/abs/2506.24108 · HTML: https://mlsanity.com/paper/2506.24108/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://mlsanity.com/paper/2305.08891/) (2023)
- [Guiding a Diffusion Model with a Bad Version of Itself](https://mlsanity.com/paper/2406.02507/) (2024)
- [A General Framework for Inference-time Scaling and Steering of Diffusion Models](https://mlsanity.com/paper/2501.06848/) (2025)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Sketch-Guided Text-to-Image Diffusion Models](https://mlsanity.com/paper/2211.13752/) (2022)
- [CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models](https://mlsanity.com/paper/2406.08070/) (2024)
- [CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling](https://mlsanity.com/paper/2310.17347/) (2023)
- [Eliminating Oversaturation and Artifacts of High Guidance Scales in Diffusion Models](https://mlsanity.com/paper/2410.02416/) (2024)
- [HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance](https://mlsanity.com/paper/2305.18766/) (2023)
- [Generating images of rare concepts using pre-trained diffusion models](https://mlsanity.com/paper/2304.14530/) (2023)
- [Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2404.05384/) (2024)
- [Good Seed Makes a Good Crop: Discovering Secret Seeds in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2405.14828/) (2024)
- [Compress Guidance in Conditional Diffusion Sampling](https://mlsanity.com/paper/2408.11194/) (2024)

## Built on by (most cited first, 11 of 11)

- [Learn to Guide Your Diffusion Model](https://mlsanity.com/paper/2510.00815/) (2025)
- [SAEdit: Token-level control for continuous image editing via Sparse AutoEncoder](https://mlsanity.com/paper/2510.05081/) (2025)
- [Continuous Control of Editing Models via Adaptive-Origin Guidance](https://mlsanity.com/paper/2602.03826/) (2026)
- [Dimension-Free Multimodal Sampling via Preconditioned Annealed Langevin Dynamics](https://mlsanity.com/paper/2602.01449/) (2026)
- [On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers](https://mlsanity.com/paper/2603.28762/) (2026)
- [AC3S: Adaptive Conditioning for 3D-Aware Synthetic Data Generation](https://mlsanity.com/paper/2606.31204/) (2026)
- [RSTR: Reducing SpatioTemporal Redundancy in Diffusion Transformers](https://mlsanity.com/paper/2512.14096/) (2025)
- [PathGuide: Dynamic Classifier-Free Guidance via On-Policy Transport Alignment](https://mlsanity.com/paper/2608.29107/) (2026)
- [Rethinking Global Text Conditioning in Diffusion Transformers](https://mlsanity.com/paper/2602.09268/) (2026)
- [FlowBender: Feedback-Aware Training for Self-Correcting Conditional Flows](https://mlsanity.com/paper/2606.20404/) (2026)
- [Semantic Browsing: Controllable Diversity for Image Generation](https://mlsanity.com/paper/2606.23679/) (2026)

## Abstract (arXiv)

Denoising diffusion models excel at generating high-quality images conditioned on text prompts, yet their effectiveness heavily relies on careful guidance during the sampling process. Classifier-Free Guidance (CFG) provides a widely used mechanism for steering generation by setting the guidance scale, which balances image quality and prompt alignment. However, the choice of the guidance scale has a critical impact on the convergence toward a visually appealing and prompt-adherent image. In this work, we propose an annealing guidance scheduler which dynamically adjusts the guidance scale over time based on the conditional noisy signal. By learning a scheduling policy, our method addresses the temperamental behavior of CFG. Empirical results demonstrate that our guidance scheduler significantly enhances image quality and alignment with the text prompt, advancing the performance of text-to-image generation. Notably, our novel scheduler requires no additional activations or memory consumption, and can seamlessly replace the common classifier-free guidance, offering an improved trade-off between prompt alignment and quality.