# One-step Diffusion Models with f-Divergence Distribution Matching

arXiv: https://arxiv.org/abs/2502.15681 · HTML: https://mlsanity.com/paper/2502.15681/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 68 of its 78 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 68)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)

## Built on by (most cited first, 30 of 43)

- [Distribution Matching Distillation Meets Reinforcement Learning](https://mlsanity.com/paper/2511.13649/) (2025)
- [Transition Matching Distillation for Fast Video Generation](https://mlsanity.com/paper/2601.09881/) (2026)
- [Flash-WAM: Modality-Aware Distillation for World Action Models](https://mlsanity.com/paper/2606.05254/) (2026)
- [Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct](https://mlsanity.com/paper/2509.25035/) (2025)
- [Fast Text-to-Audio Generation with Adversarial Post-Training](https://mlsanity.com/paper/2505.08175/) (2025)
- [Flow Map Distillation Without Data](https://mlsanity.com/paper/2511.19428/) (2025)
- [Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instruction](https://mlsanity.com/paper/2505.20755/) (2025)
- [Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation](https://mlsanity.com/paper/2604.23632/) (2026)
- [Mode Seeking meets Mean Seeking for Fast Long Video Generation](https://mlsanity.com/paper/2602.24289/) (2026)
- [Parallel Decoding Distillation for Fast Image and Video Generation](https://mlsanity.com/paper/2607.26004/) (2026)
- [Data-Forcing Distillation: Restoring Diversity and Fidelity in Few-Step Video Generation](https://mlsanity.com/paper/2606.18478/) (2026)
- [Score Distillation of Flow Matching Models](https://mlsanity.com/paper/2509.25127/) (2025)
- [Riemannian Consistency Model](https://mlsanity.com/paper/2510.00983/) (2025)
- [Skywork UniPic 3.0: Unified Multi-Image Composition via Sequence Modeling](https://mlsanity.com/paper/2601.15664/) (2026)
- [One-Step Flow Policy: Self-Distillation for Fast Visuomotor Policies](https://mlsanity.com/paper/2603.12480/) (2026)
- [Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation](https://mlsanity.com/paper/2603.21864/) (2026)
- [Stable Audio 3](https://mlsanity.com/paper/2605.17991/) (2026)
- [Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generator](https://mlsanity.com/paper/2503.15457/) (2025)
- [PyramidalWan: On Making Pretrained Video Model Pyramidal for Efficient Inference](https://mlsanity.com/paper/2601.04792/) (2026)
- [Midpoint Generative Models](https://mlsanity.com/paper/2605.29920/) (2026)
- [DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation](https://mlsanity.com/paper/2607.26811/) (2026)
- [From Structure to Detail: Hierarchical Distillation for Efficient Diffusion Model](https://mlsanity.com/paper/2511.08930/) (2025)
- [Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster](https://mlsanity.com/paper/2605.18204/) (2026)
- [CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation](https://mlsanity.com/paper/2606.21982/) (2026)
- [Toward Theoretical Insights into Diffusion Trajectory Distillation via Operator Merging](https://mlsanity.com/paper/2505.16024/) (2025)
- [A Unified Framework for Diffusion Model Unlearning with f-Divergence](https://mlsanity.com/paper/2509.21167/) (2025)
- [Ultra Fast PDE Solving via Physics Guided Few-step Diffusion](https://mlsanity.com/paper/2602.03627/) (2026)
- [Accelerating Speculative Diffusions via Block Verification](https://mlsanity.com/paper/2606.13426/) (2026)
- [Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers](https://mlsanity.com/paper/2606.32020/) (2026)
- [FVGen: Accelerating Novel-View Synthesis with Adversarial Video Diffusion Distillation](https://mlsanity.com/paper/2508.06392/) (2025)

## Abstract (arXiv)

Sampling from diffusion models involves a slow iterative process that hinders their practical deployment, especially for interactive applications. To accelerate generation speed, recent approaches distill a multi-step diffusion model into a single-step student generator via variational score distillation, which matches the distribution of samples generated by the student to the teacher's distribution. However, these approaches use the reverse Kullback-Leibler (KL) divergence for distribution matching which is known to be mode seeking. In this paper, we generalize the distribution matching approach using a novel f-divergence minimization framework, termed f-distill, that covers different divergences with different trade-offs in terms of mode coverage and training variance. We derive the gradient of the f-divergence between the teacher and student distributions and show that it is expressed as the product of their score differences and a weighting function determined by their density ratio. This weighting function naturally emphasizes samples with higher density in the teacher distribution, when using a less mode-seeking divergence. We observe that the popular variational score distillation approach using the reverse-KL divergence is a special case within our framework. Empirically, we demonstrate that alternative f-divergences, such as forward-KL and Jensen-Shannon divergences, outperform the current best variational score distillation methods across image generation tasks. In particular, when using Jensen-Shannon divergence, f-distill achieves current state-of-the-art one-step generation performance on ImageNet64 and zero-shot text-to-image generation on MS-COCO. Project page: https://research.nvidia.com/labs/genair/f-distill