mlsanityData to 7 Oct 2026

Guiding a Diffusion Model with a Bad Version of Itself

arXiv 2406.02507 ↗First posted 4 Jun 2024cs.CV cs.AI cs.LGcited by 286

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The primary axes of interest in image-generating diffusion models are image quality, the amount of variation in the results, and how well the results align with a given condition, e.g., a class label or a text prompt. The popular classifier-free guidance approach uses an unconditional model to guide a conditional model, leading to simultaneously better prompt alignment and higher-quality images at the cost of reduced variation. These effects seem inherently entangled, and thus hard to control. We make the surprising observation that it is possible to obtain disentangled control over image quality without compromising the amount of variation by guiding generation using a smaller, less-trained version of the model itself rather than an unconditional model. This leads to significant improvements in ImageNet generation, setting record FIDs of 1.01 for 64x64 and 1.25 for 512x512, using publicly available networks. Furthermore, the method is also applicable to unconditional diffusion models, drastically improving their quality.

Key facts. Cited by 286 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 48 of its 57 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

48 of 57 references.

Closest ancestors · 25

Same direction, newest first

  1. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Models22 Apr
  2. 2024Analysis of Classifier-Free Guidance Weight Schedulers19 Apr
  3. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models11 Apr
  4. 2024Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance26 Mar
  5. 2024Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers21 Jan
  6. 2023Characteristic Guidance: Non-linear Correction for Diffusion Model at Large Guidance Scale11 Dec
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  8. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling26 Oct
  9. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  10. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  11. 2022Scalable Diffusion Models with Transformers19 Dec
  12. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
13 more
  1. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidance3 Oct
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  6. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Implicit Models6 Oct
  13. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 23

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
11 more
  1. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  2. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  3. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  4. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  5. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  6. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  7. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  8. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  9. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  10. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 62
  11. 2017Megapixel Size Image Creation using Generative Adversarial Networkscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 147; 2026: 117.

286 papers.

Sort:

By area:

  1. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  2. 2025Diffusion Adversarial Post-Training for One-Step Video Generationcited by 142
  3. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  4. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Expertscited by 85
  5. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  6. 2024Classifier-Free Guidance is a Predictor-Correctorcited by 79
  7. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusioncited by 74
  8. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
  9. 2024A Noise is Worth Diffusion Guidancecited by 48
  10. 2025GenMol: A Drug Discovery Generalist with Discrete Diffusioncited by 47
  11. 2025CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Modelscited by 47
  12. 2024What does guidance do? A fine-grained analysis in a simple settingcited by 45
38 more
  1. 2025One-step Diffusion Models with f-Divergence Distribution Matchingcited by 43
  2. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 42
  3. 2024Language-Guided Image Tokenization for Generationcited by 36
  4. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Modelscited by 35
  5. 2024Self-Improving Diffusion Models with Synthetic Datacited by 34
  6. 2024Simple ReFlow: Improved Techniques for Fast Flow Modelscited by 34
  7. 2024Spatiotemporal Skip Guidance for Enhanced Video Diffusion Samplingcited by 33
  8. 2026Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generationcited by 33
  9. 2025STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesiscited by 31
  10. 2024Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budgetcited by 30
  11. 2025Accelerated Diffusion Models via Speculative Samplingcited by 30
  12. 2025Contrastive Flow Matchingcited by 29
  13. 2025Diffusion Models without Classifier-free Guidancecited by 28
  14. 2024Flowing from Words to Pixels: A Noise-Free Framework for Cross-Modality Evolutioncited by 26
  15. 2025FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editingcited by 24
  16. 2026Improved Baselines with Representation Autoencoderscited by 24
  17. 2024Self-Guidance: Boosting Flow and Diffusion Generation on Their Owncited by 22
  18. 2025Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminatorcited by 22
  19. 2025Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Modelscited by 21
  20. 2024Importance-Based Token Merging for Efficient Image and Video Generationcited by 20
  21. 2024Diffusion Models as Cartoonists: The Curious Case of High Density Regionscited by 19
  22. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  23. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 19
  24. 2024Rectified Diffusion Guidance for Conditional Generationcited by 18
  25. 2025Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlocited by 18
  26. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  27. 2024ETTA: Elucidating the Design Space of Text-to-Audio Modelscited by 17
  28. 2025Visual Generation Without Guidancecited by 17
  29. 2025Gaussian Mixture Flow Matching Modelscited by 16
  30. 2025Feedback Guidance of Diffusion Modelscited by 16
  31. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusioncited by 16
  32. 2025Token Perturbation Guidance for Diffusion Modelscited by 14
  33. 2025Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Controlcited by 14
  34. 2025Rectified-CFG++ for Flow Based Modelscited by 14
  35. 2025Progressive Tempering Sampler with Diffusioncited by 13
  36. 2025Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scalescited by 13
  37. 2025Flow Map Distillation Without Datacited by 13
  38. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12

50 most cited of 286 All 286 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Towards Practical Plug-and-Play Diffusion Models
  2. 2022On Distillation of Guided Diffusion Models
  3. 2023Elucidating The Design Space of Classifier-Guided Diffusion Generation
  4. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance
  5. 2024Plug-and-Play Diffusion Distillation
  6. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  7. 2025In-situ Autoguidance: Eliciting Self-Correction in Diffusion Models
  8. 2026Correcting Guided Diffusion Trajectories with Spectral Alignment