mlsanityData to 14 Oct 2026

VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation

arXiv 2508.20646 ↗First posted 28 Aug 2025cs.LGcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recently, diffusion distillation methods have compressed thousand-step teacher diffusion models into one-step student generators while preserving sample quality. Most existing approaches train the student model using a diffusive divergence whose gradient is approximated via the student's score function, learned through denoising score matching (DSM). Since DSM training is imperfect, the resulting gradient estimate is inevitably biased, leading to sub-optimal performance. In this paper, we propose VarDiU (pronounced /va:rdju:/), a Variational Diffusive Upper Bound that admits an unbiased gradient estimator and can be directly applied to diffusion distillation. Using this objective, we compare our method with Diff-Instruct and demonstrate that it achieves higher generation quality and enables a more efficient and stable training procedure for one-step diffusion distillation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 47 of its 59 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

47 of 59 references.

Closest ancestors · 32

Same direction, newest first

  1. 2025Inductive Moment Matching10 Mar
  2. 2025One-step Diffusion Models with f-Divergence Distribution Matching21 Feb
  3. 2025DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision11 Feb
  4. 2025Distributional Diffusion Models with Scoring Rules4 Feb
  5. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Step19 Oct
  6. 2024Multistep Distillation of Diffusion Models via Moment Matching6 Jun
  7. 2024EM Distillation for One-step Diffusion Models27 May
  8. 2024Improved Distribution Matching Distillation for Fast Image Synthesis23 May
  9. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation5 Apr
  10. 2024Bidirectional Consistency Models26 Mar
  11. 2024Multistep Consistency Models11 Mar
  12. 2023Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration26 Oct
20 more
  1. 2023Improved Techniques for Training Consistency Models22 Oct
  2. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  3. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion1 Oct
  4. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  5. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  6. 2023Consistency Models2 Mar
  7. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  8. 2022On Distillation of Guided Diffusion Models6 Oct
  9. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  13. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  14. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  15. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  16. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  17. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  18. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  19. 2020Denoising Diffusion Implicit Models6 Oct
  20. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 15

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2015Variational Inference with Normalizing Flowscited by 2,170
  6. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  7. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  8. 2019Neural Spline Flowscited by 596
  9. 2022Flow Annealed Importance Sampling Bootstrapcited by 127
  10. 2024Normalizing Flows are Capable Generative Modelscited by 91
  11. 2023normflows: A PyTorch Package for Normalizing Flowscited by 56
  12. 2017Variational Inference using Implicit Distributionscited by 39
3 more
  1. 2018Spread Divergencecited by 23
  2. 2024Training Neural Samplers with Reverse Diffusive KL Divergencecited by 21
  3. 2019Variational f-divergence Minimizationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Diffusion Alignment Beyond KL: Variance Minimisation as Effective Policy Optimisercited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025ReDiF: Reinforced Distillation for Few Step Diffusion
  2. 2023One-Step Diffusion Distillation via Deep Equilibrium Models
  3. 2024DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning
  4. 2026Consistent Distribution Matching for Data-Free Diffusion Distillation
  5. 2024Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation
  6. 2025Understanding Generalization in Diffusion Distillation via Probability Flow Distance
  7. 2026A Quantitative Approximation Framework for Flow Distillation in Diffusion Models
  8. 2024Inference-Time Diffusion Model Distillation