# DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision arXiv: https://arxiv.org/abs/2502.08005 ยท HTML: https://mlsanity.com/paper/2502.08005/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 52 of its 69 references that are in the map. - First posted on arXiv 11 Feb 2025. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 52) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022) - [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017) - [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022) - [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023) - [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018) - [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023) - [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022) - [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022) - [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024) - [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018) - [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023) - [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021) - [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017) - [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023) - [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023) - [Analyzing and Improving the Training Dynamics of Diffusion Models](https://mlsanity.com/paper/2312.02696/) (2023) - [One Step Diffusion via Shortcut Models](https://mlsanity.com/paper/2410.12557/) (2024) - [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022) - [Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models](https://mlsanity.com/paper/2410.11081/) (2024) - [Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models](https://mlsanity.com/paper/2305.18455/) (2023) - [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024) - [Diffusion-GAN: Training GANs with Diffusion](https://mlsanity.com/paper/2206.02262/) (2022) ## Built on by (most cited first, 9 of 9) - [Few-Step Diffusion via Score identity Distillation](https://mlsanity.com/paper/2505.12674/) (2025) - [Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generator](https://mlsanity.com/paper/2503.15457/) (2025) - [Midpoint Generative Models](https://mlsanity.com/paper/2605.29920/) (2026) - [Distilled Decoding 2: One-step Sampling of Image Auto-regressive Models with Conditional Score Distillation](https://mlsanity.com/paper/2510.21003/) (2025) - [VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation](https://mlsanity.com/paper/2508.20646/) (2025) - [Diffusion Alignment Beyond KL: Variance Minimisation as Effective Policy Optimiser](https://mlsanity.com/paper/2602.12229/) (2026) - [Diffusion Distillation for Efficient Weather Ensembles](https://mlsanity.com/paper/2608.27728/) (2026) - [Controlling Dependence in Implicit Generative Models via Spread Mutual Information](https://mlsanity.com/paper/2610.10021/) (2026) - [Are We Overthinking One-Step Generation?](https://mlsanity.com/paper/2610.20331/) (2026) ## Abstract (arXiv) Score-based distillation methods (e.g., variational score distillation) train one-step diffusion models by first pre-training a teacher score model and then distilling it into a one-step student model. However, the gradient estimator in the distillation stage usually suffers from two sources of bias: (1) biased teacher supervision due to score estimation error incurred during pre-training, and (2) the student model's score estimation error during distillation. These biases can degrade the quality of the resulting one-step diffusion model. To address this, we propose DiffRatio, a new framework for training one-step diffusion models: instead of estimating the teacher and student scores independently and then taking their difference, we directly estimate the score difference as the gradient of a learned log density ratio between the student and data distributions across diffusion time steps. This approach greatly simplifies the training pipeline, significantly reduces gradient estimation bias, and improves one-step generation quality. Additionally, it also reduces auxiliary network size by using a lightweight density-ratio network instead of two full score networks, which improves computational and memory efficiency. DiffRatio achieves competitive one-step generation results on CIFAR-10 and ImageNet (64x64 and 512x512), outperforming most teacher-supervised distillation methods. Moreover, the learned density ratio naturally serves as a verifier, enabling a principled inference-time parallel scaling scheme that further improves the generation quality without external rewards or additional sequential computation.