# SwiftBrush v2: Make Your One-step Diffusion Model Better Than Its Teacher

arXiv: https://arxiv.org/abs/2408.14176 · HTML: https://mlsanity.com/paper/2408.14176/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 44 of its 53 references that are in the map.
- First posted on arXiv 26 Aug 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 44)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time](https://mlsanity.com/paper/2203.05482/) (2022)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [CogView: Mastering Text-to-Image Generation via Transformers](https://mlsanity.com/paper/2105.13290/) (2021)
- [Muse: Text-To-Image Generation via Masked Generative Transformers](https://mlsanity.com/paper/2301.00704/) (2023)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Dataless Knowledge Fusion by Merging Weights of Language Models](https://mlsanity.com/paper/2212.09849/) (2022)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)

## Built on by (most cited first, 30 of 45)

- [Adversarial Diffusion Compression for Real-World Image Super-Resolution](https://mlsanity.com/paper/2411.13383/) (2024)
- [One-step Diffusion Models with f-Divergence Distribution Matching](https://mlsanity.com/paper/2502.15681/) (2025)
- [SwiftEdit: Lightning Fast Text-Guided Image Editing via One-Step Diffusion](https://mlsanity.com/paper/2412.04301/) (2024)
- [SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device](https://mlsanity.com/paper/2412.10494/) (2024)
- [One-Step Diffusion-Based Image Compression with Semantic Distillation](https://mlsanity.com/paper/2505.16687/) (2025)
- [ChordEdit: One-Step Low-Energy Transport for Image Editing](https://mlsanity.com/paper/2602.19083/) (2026)
- [NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training](https://mlsanity.com/paper/2412.02030/) (2024)
- [Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generation](https://mlsanity.com/paper/2412.16906/) (2024)
- [Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models](https://mlsanity.com/paper/2505.21179/) (2025)
- [FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency](https://mlsanity.com/paper/2506.08822/) (2025)
- [Few-Step Diffusion via Score identity Distillation](https://mlsanity.com/paper/2505.12674/) (2025)
- [SD3.5-Flash: Distribution-Guided Distillation of Generative Flows](https://mlsanity.com/paper/2509.21318/) (2025)
- [Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform](https://mlsanity.com/paper/2505.11892/) (2025)
- [Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds](https://mlsanity.com/paper/2507.13343/) (2025)
- [Supercharged One-step Text-to-Image Diffusion Models with Negative Prompts](https://mlsanity.com/paper/2412.02687/) (2024)
- [Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse](https://mlsanity.com/paper/2505.16284/) (2025)
- [Anti-I2V: Safeguarding your photos from malicious image-to-video generation](https://mlsanity.com/paper/2603.24570/) (2026)
- [CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models](https://mlsanity.com/paper/2507.13984/) (2025)
- [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025)
- [Improved Training Technique for Shortcut Models](https://mlsanity.com/paper/2510.21250/) (2025)
- [Human Body Restoration with One-Step Diffusion Model and A New Benchmark](https://mlsanity.com/paper/2502.01411/) (2025)
- [One-Way Ticket:Time-Independent Unified Encoder for Distilling Text-to-Image Diffusion Models](https://mlsanity.com/paper/2505.21960/) (2025)
- [FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation](https://mlsanity.com/paper/2412.16915/) (2024)
- [MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection](https://mlsanity.com/paper/2505.19149/) (2025)
- [FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editing](https://mlsanity.com/paper/2509.22244/) (2025)
- [One-Step Residual Shifting Diffusion for Image Super-Resolution via Distillation](https://mlsanity.com/paper/2503.13358/) (2025)
- [From Cradle to Cane: A Two-Pass Framework for High-Fidelity Lifespan Face Aging](https://mlsanity.com/paper/2506.20977/) (2025)
- [Fast-SAM3D: 3Dfy Anything in Images but Faster](https://mlsanity.com/paper/2602.05293/) (2026)
- [Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability](https://mlsanity.com/paper/2505.03097/) (2025)
- [InverFill: One-Step Inversion for Enhanced Few-Step Diffusion Inpainting](https://mlsanity.com/paper/2603.23463/) (2026)

## Abstract (arXiv)

In this paper, we aim to enhance the performance of SwiftBrush, a prominent one-step text-to-image diffusion model, to be competitive with its multi-step Stable Diffusion counterpart. Initially, we explore the quality-diversity trade-off between SwiftBrush and SD Turbo: the former excels in image diversity, while the latter excels in image quality. This observation motivates our proposed modifications in the training methodology, including better weight initialization and efficient LoRA training. Moreover, our introduction of a novel clamped CLIP loss enhances image-text alignment and results in improved image quality. Remarkably, by combining the weights of models trained with efficient LoRA and full training, we achieve a new state-of-the-art one-step diffusion model, achieving an FID of 8.14 and surpassing all GAN-based and multi-step Stable Diffusion models. The project page is available at https://swiftbrushv2.github.io.