# Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models

arXiv: https://arxiv.org/abs/2312.12487 · HTML: https://mlsanity.com/paper/2312.12487/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 56 of its 63 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 56)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)

## Built on by (most cited first, 30 of 31)

- [Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models](https://mlsanity.com/paper/2404.07724/) (2024)
- [Training-free and Adaptive Sparse Attention for Efficient Long Video Generation](https://mlsanity.com/paper/2502.21079/) (2025)
- [Analysis of Classifier-Free Guidance Weight Schedulers](https://mlsanity.com/paper/2404.13040/) (2024)
- [Faster Diffusion via Temporal Attention Decomposition](https://mlsanity.com/paper/2404.02747/) (2024)
- [Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation](https://mlsanity.com/paper/2405.05224/) (2024)
- [Dynamic Negative Guidance of Diffusion Models](https://mlsanity.com/paper/2410.14398/) (2024)
- [Stage-wise Dynamics of Classifier-Free Guidance in Diffusion Models](https://mlsanity.com/paper/2509.22007/) (2025)
- [Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps](https://mlsanity.com/paper/2406.14539/) (2024)
- [FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute](https://mlsanity.com/paper/2502.20126/) (2025)
- [Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation](https://mlsanity.com/paper/2402.18839/) (2024)
- [Coarse-to-Real: Generative Rendering for Populated Dynamic Scenes](https://mlsanity.com/paper/2601.22301/) (2026)
- [Efficient Distillation of Classifier-Free Guidance using Adapters](https://mlsanity.com/paper/2503.07274/) (2025)
- [MatchDiffusion: Training-free Generation of Match-cuts](https://mlsanity.com/paper/2411.18677/) (2024)
- [Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment](https://mlsanity.com/paper/2508.06160/) (2025)
- [Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Models](https://mlsanity.com/paper/2506.10177/) (2025)
- [Tortoise and Hare Guidance: Accelerating Diffusion Model Inference with Multirate Integration](https://mlsanity.com/paper/2511.04117/) (2025)
- [Animated Stickers: Bringing Stickers to Life with Video Diffusion](https://mlsanity.com/paper/2402.06088/) (2024)
- [How Much To Guide: Revisiting Adaptive Guidance in Classifier-Free Guidance Text-to-Vision Diffusion Models](https://mlsanity.com/paper/2506.08351/) (2025)
- [Improved Sample Complexity For Diffusion Model Training Without Empirical Risk Minimizer Access](https://mlsanity.com/paper/2505.18344/) (2025)
- [DS-Diffusion: Data Style-Guided Diffusion Model for Time-Series Generation](https://mlsanity.com/paper/2509.18584/) (2025)
- [Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures](https://mlsanity.com/paper/2605.17850/) (2026)
- [RSTR: Reducing SpatioTemporal Redundancy in Diffusion Transformers](https://mlsanity.com/paper/2512.14096/) (2025)
- [Joint Residual Reweighting for Classifier Free Guidance in Flow-Matching Zero-Shot TTS](https://mlsanity.com/paper/2606.25672/) (2026)
- [PathGuide: Dynamic Classifier-Free Guidance via On-Policy Transport Alignment](https://mlsanity.com/paper/2608.29107/) (2026)
- [LAMS-Edit: Latent and Attention Mixing with Schedulers for Improved Content Preservation in Diffusion-Based Image and Style Editing](https://mlsanity.com/paper/2601.02987/) (2026)
- [DICT: Data Injection and Contrastive Trajectory Refinement for Conditional Image Generation with Diffusion Models](https://mlsanity.com/paper/2607.03899/) (2026)
- [Adversarial Learning of Classifier-Free Guidance Schedules](https://mlsanity.com/paper/2608.14038/) (2026)
- [Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generation](https://mlsanity.com/paper/2609.19702/) (2026)
- [FastGuide: Accelerating Reward Guidance for Diffusion Large Language Models](https://mlsanity.com/paper/2609.36202/) (2026)
- [TRAC: Trajectory-aware Reuse and Adaptive Correction for Efficient Autoregressive Video Generation](https://mlsanity.com/paper/2610.02779/) (2026)

## Abstract (arXiv)

This paper presents a comprehensive study on the role of Classifier-Free Guidance (CFG) in text-conditioned diffusion models from the perspective of inference efficiency. In particular, we relax the default choice of applying CFG in all diffusion steps and instead search for efficient guidance policies. We formulate the discovery of such policies in the differentiable Neural Architecture Search framework. Our findings suggest that the denoising steps proposed by CFG become increasingly aligned with simple conditional steps, which renders the extra neural network evaluation of CFG redundant, especially in the second half of the denoising process. Building upon this insight, we propose "Adaptive Guidance" (AG), an efficient variant of CFG, that adaptively omits network evaluations when the denoising process displays convergence. Our experiments demonstrate that AG preserves CFG's image quality while reducing computation by 25%. Thus, AG constitutes a plug-and-play alternative to Guidance Distillation, achieving 50% of the speed-ups of the latter while being training-free and retaining the capacity to handle negative prompts. Finally, we uncover further redundancies of CFG in the first half of the diffusion process, showing that entire neural function evaluations can be replaced by simple affine transformations of past score estimates. This method, termed LinearAG, offers even cheaper inference at the cost of deviating from the baseline model. Our findings provide insights into the efficiency of the conditional denoising process that contribute to more practical and swift deployment of text-conditioned diffusion models.