mlsanityData to 7 Oct 2026

SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation

arXiv 2312.05239 ↗First posted 8 Dec 2023cs.CVcited by 109

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Despite their ability to generate high-resolution and diverse images from text prompts, text-to-image diffusion models often suffer from slow iterative sampling processes. Model distillation is one of the most effective directions to accelerate these models. However, previous distillation methods fail to retain the generation quality while requiring a significant amount of images for training, either from real data or synthetically generated by the teacher model. In response to this limitation, we present a novel image-free distillation scheme named SwiftBrush. Drawing inspiration from text-to-3D synthesis, in which a 3D neural radiance field that aligns with the input prompt can be obtained from a 2D text-to-image diffusion prior via a specialized loss without the use of any 3D data ground-truth, our approach re-purposes that same loss for distilling a pretrained multi-step text-to-image model to a student network that can generate high-fidelity images with just a single inference step. In spite of its simplicity, our model stands as one of the first one-step text-to-image generators that can produce images of comparable quality to Stable Diffusion without reliance on any training image data. Remarkably, SwiftBrush achieves an FID score of 16.67 and a CLIP score of 0.29 on the COCO-30K benchmark, achieving competitive results or even substantially surpassing existing state-of-the-art distillation techniques.

Key facts. Cited by 109 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 43 of its 49 references that are in the map. First posted on arXiv 8 Dec 2023.

Builds on

43 of 49 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  3. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  4. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
25 more
  1. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  2. 2022Video Diffusion Modelscited by 2,280
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  4. 2023Consistency Modelscited by 1,778
  5. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  6. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  7. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  9. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  10. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  11. 2022On Distillation of Guided Diffusion Modelscited by 721
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  13. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  14. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  15. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  16. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  17. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  18. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  19. 2022Novel View Synthesis with Diffusion Modelscited by 299
  20. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  21. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Modelscited by 232
  22. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  23. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  24. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrappingcited by 96
  25. 2023Unifying GANs and Score-Based Diffusion as Generative Particle Modelscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 39; 2025: 41; 2026: 28.

109 papers.

Sort:

By area:

  1. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  2. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  3. 2024Consistency Models Made Easycited by 140
  4. 2024PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Acceleratorcited by 95
  5. 2024TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolutioncited by 83
  6. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 76
  7. 2024One-Step Diffusion Distillation through Score Implicit Matchingcited by 68
  8. 2024EM Distillation for One-step Diffusion Modelscited by 60
  9. 2025Learning Few-Step Diffusion Models by Trajectory Distribution Matchingcited by 49
  10. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  11. 2024SwiftBrush v2: Make Your One-step Diffusion Model Better Than Its Teachercited by 45
  12. 2025One-step Diffusion Models with f-Divergence Distribution Matchingcited by 43
38 more
  1. 2025One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillationcited by 40
  2. 2024Reward Guided Latent Consistency Distillationcited by 35
  3. 2023Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inferencecited by 33
  4. 2025FlashVideo: Flowing Fidelity to Detail for Efficient High-Resolution Video Generationcited by 32
  5. 2024Simple and Fast Distillation of Diffusion Modelscited by 31
  6. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 30
  7. 2024SwiftEdit: Lightning Fast Text-Guided Image Editing via One-Step Diffusioncited by 28
  8. 2024Flow Generator Matchingcited by 26
  9. 2024SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditionscited by 23
  10. 2025One-Step Diffusion-Based Image Compression with Semantic Distillationcited by 23
  11. 2024Score Forgetting Distillation: A Swift, Data-Free Method for Machine Unlearning in Diffusion Modelscited by 22
  12. 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillationcited by 21
  13. 2024Lazy Diffusion Transformer for Interactive Image Editingcited by 19
  14. 2024Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminatorcited by 18
  15. 2025Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learningcited by 17
  16. 2024Epsilon-VAE: Denoising as Visual Decodingcited by 16
  17. 2024NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Trainingcited by 16
  18. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 15
  19. 2024SCott: Accelerating Diffusion Models with Stochastic Consistency Distillationcited by 14
  20. 2024Turbo3D: Ultra-fast Text-to-3D Generationcited by 14
  21. 2024Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generationcited by 14
  22. 2025NTIRE 2025 Challenge on Efficient Burst HDR and Restoration: Datasets, Methods, and Resultscited by 14
  23. 2025UltraVSR: Achieving Ultra-Realistic Video Super-Resolution with Efficient One-Step Diffusion Spacecited by 14
  24. 2024Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulationcited by 13
  25. 2025Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instructioncited by 13
  26. 2025FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistencycited by 13
  27. 2025Flow Map Distillation Without Datacited by 13
  28. 2025RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectoriescited by 12
  29. 2024Aligning Few-Step Diffusion Models with Dense Reward Difference Learningcited by 11
  30. 2024Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generationcited by 10
  31. 2025Few-Step Diffusion via Score identity Distillationcited by 10
  32. 2024Multi-student Diffusion Distillation for Better One-step Generatorscited by 9
  33. 2025InterLCM: Low-Quality Images as Intermediate States of Latent Consistency Models for Effective Blind Face Restorationcited by 9
  34. 2025TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Visioncited by 9
  35. 2025One-Step Offline Distillation of Diffusion-based Models via Koopman Modelingcited by 9
  36. 2025SD3.5-Flash: Distribution-Guided Distillation of Generative Flowscited by 9
  37. 2024David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-trainingcited by 8
  38. 2024Supercharged One-step Text-to-Image Diffusion Models with Negative Promptscited by 8

50 most cited of 109 All 109 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Text-Image Conditioned Diffusion for Consistent Text-to-3D Generation
  2. 2025Few-Step Distillation for Text-to-Image Generation: A Practical Guide
  3. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
  4. 2023PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion
  5. 2026i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models
  6. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  7. 2024ScaleDreamer: Scalable Text-to-3D Synthesis with Asynchronous Score Distillation
  8. 2024A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D