# SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow

arXiv: https://arxiv.org/abs/2407.12718 · HTML: https://mlsanity.com/paper/2407.12718/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 60 of its 69 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 60)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)

## Built on by (most cited first, 30 of 30)

- [Adversarial Diffusion Compression for Real-World Image Super-Resolution](https://mlsanity.com/paper/2411.13383/) (2024)
- [Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield](https://mlsanity.com/paper/2511.22677/) (2025)
- [GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling](https://mlsanity.com/paper/2501.18898/) (2025)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [Mobile Video Diffusion](https://mlsanity.com/paper/2412.07583/) (2024)
- [On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices](https://mlsanity.com/paper/2502.04363/) (2025)
- [X2I: Seamless Integration of Multimodal Understanding into Diffusion Transformer via Attention Distillation](https://mlsanity.com/paper/2503.06134/) (2025)
- [SparseDiT: Token Sparsification for Efficient Diffusion Transformer](https://mlsanity.com/paper/2412.06028/) (2024)
- [PocketSR: The Super-Resolution Expert in Your Pocket Mobiles](https://mlsanity.com/paper/2510.03012/) (2025)
- [SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories](https://mlsanity.com/paper/2502.16972/) (2025)
- [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025)
- [VideoFlexTok: Flexible-Length Coarse-to-Fine Video Tokenization](https://mlsanity.com/paper/2604.12887/) (2026)
- [A Survey on Pre-Trained Diffusion Model Distillations](https://mlsanity.com/paper/2502.08364/) (2025)
- [Fine-Tuning Diffusion Models via Intermediate Distribution Shaping](https://mlsanity.com/paper/2510.02692/) (2025)
- [Balanced conic rectified flow](https://mlsanity.com/paper/2510.25229/) (2025)
- [SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flow](https://mlsanity.com/paper/2504.07776/) (2025)
- [Upcycling Text-to-Image Diffusion Models for Multi-Task Capabilities](https://mlsanity.com/paper/2503.11905/) (2025)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [Hierarchical Koopman Diffusion: Fast Generation with Interpretable Diffusion Trajectory](https://mlsanity.com/paper/2510.12220/) (2025)
- [Eliminating VAE for Fast and High-Resolution Generative Detail Restoration](https://mlsanity.com/paper/2602.10630/) (2026)
- [Toward Early Quality Assessment of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2603.02829/) (2026)
- [Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation](https://mlsanity.com/paper/2408.15991/) (2024)
- [One-step Diffusion Models with Bregman Density Ratio Matching](https://mlsanity.com/paper/2510.16983/) (2025)
- [Habilis-β: A Fast-Motion and Long-Lasting On-Device Vision-Language-Action Model](https://mlsanity.com/paper/2602.18813/) (2026)
- [On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices](https://mlsanity.com/paper/2503.23796/) (2025)
- [ReSeFlow: Rectifying SE(3)-Equivariant Policy Learning Flows](https://mlsanity.com/paper/2509.22695/) (2025)
- [Align Your Tangent: Training Better Consistency Models via Manifold-Aligned Tangents](https://mlsanity.com/paper/2510.00658/) (2025)
- [SymDrift: One-Shot Generative Modeling under Symmetries](https://mlsanity.com/paper/2605.06140/) (2026)
- [ReFPO: Reflow Regularization for Flow Matching Policy Gradients](https://mlsanity.com/paper/2606.21086/) (2026)
- [IB-Flow: Information Bottleneck-Guided CFG Distillation for Few-Step Text-to-Image Generation](https://mlsanity.com/paper/2607.09133/) (2026)

## Abstract (arXiv)

Diffusion models excel in high-quality generation but suffer from slow inference due to iterative sampling. While recent methods have successfully transformed diffusion models into one-step generators, they neglect model size reduction, limiting their applicability in compute-constrained scenarios. This paper aims to develop small, efficient one-step diffusion models based on the powerful rectified flow framework, by exploring joint compression of inference steps and model size. The rectified flow framework trains one-step generative models using two operations, reflow and distillation. Compared with the original framework, squeezing the model size brings two new challenges: (1) the initialization mismatch between large teachers and small students during reflow; (2) the underperformance of naive distillation on small student models. To overcome these issues, we propose Annealing Reflow and Flow-Guided Distillation, which together comprise our SlimFlow framework. With our novel framework, we train a one-step diffusion model with an FID of 5.02 and 15.7M parameters, outperforming the previous state-of-the-art one-step diffusion model (FID=6.47, 19.4M parameters) on CIFAR10. On ImageNet 64×64 and FFHQ 64×64, our method yields small one-step diffusion models that are comparable to larger models, showcasing the effectiveness of our method in creating compact, efficient one-step diffusion models.