mlsanityData to 14 Oct 2026

Optimizing DDPM Sampling with Shortcut Fine-Tuning

arXiv 2301.13362 ↗First posted 31 Jan 2023cs.LGcited by 81

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

In this study, we propose Shortcut Fine-Tuning (SFT), a new approach for addressing the challenge of fast sampling of pretrained Denoising Diffusion Probabilistic Models (DDPMs). SFT advocates for the fine-tuning of DDPM samplers through the direct minimization of Integral Probability Metrics (IPM), instead of learning the backward diffusion process. This enables samplers to discover an alternative and more efficient sampling shortcut, deviating from the backward diffusion process. Inspired by a control perspective, we propose a new algorithm SFT-PG: Shortcut Fine-Tuning with Policy Gradient, and prove that under certain assumptions, gradient descent of diffusion models with respect to IPM is equivalent to performing policy gradient. To our best knowledge, this is the first attempt to utilize reinforcement learning (RL) methods to train diffusion models. Through empirical evaluation, we demonstrate that our fine-tuning method can further enhance existing fast DDPM samplers, resulting in sample quality comparable to or even surpassing that of the full-step model across various datasets.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 53 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

37 of 53 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023Fast Inference in Denoising Diffusion Models via MMD Finetuning19 Jan
  2. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance13 Dec
  3. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  5. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  6. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  7. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  8. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  9. 2021Bilateral Denoising Diffusion Models26 Aug
  10. 2021Non Gaussian Denoising Diffusion Models14 Jun
  11. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  12. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
8 more
  1. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  2. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  3. 2021Noise Estimation for Generative Diffusion Models6 Apr
  4. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  5. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 17

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2017Improved Training of Wasserstein GANscited by 2,235
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  6. 2018Demystifying MMD GANscited by 1,293
  7. 2018Which Training Methods for GANs do actually Converge?cited by 679
  8. 2018Behavioral Cloning from Observationcited by 492
  9. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
  10. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
  11. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  12. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 254
5 more
  1. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  2. 2017IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Modelscited by 75
  3. 2019RL-GAN-Net: A Reinforcement Learning Agent Controlled GAN Network for Real-Time Point Cloud Shape Completioncited by 58
  4. 2019Model-Based Reinforcement Learning with Adversarial Training for Online Recommendationcited by 33
  5. 2018On gradient regularizers for MMD GANscited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 26; 2025: 26; 2026: 18.

81 papers.

Sort:

By area:

  1. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  2. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  3. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
  4. 2024Diffusion Policy Policy Optimizationcited by 280
  5. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Modelcited by 252
  6. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  7. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 101
  8. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 90
  9. 2023InstructVideo: Instructing Video Diffusion Models with Human Feedbackcited by 86
  10. 2024Policy Decorator: Model-Agnostic Online Refinement for Large Policy Modelcited by 70
  11. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  12. 2024A Dense Reward View on Aligning Text-to-Image Diffusion with Preferencecited by 55
38 more
  1. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 46
  2. 2023A Comprehensive Survey on Knowledge Distillation of Diffusion Modelscited by 45
  3. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 44
  4. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Modelscited by 42
  5. 2024Fine-tuning of diffusion models via stochastic control: entropy regularization and beyondcited by 42
  6. 2024Learning to Discretize Denoising Diffusion ODEscited by 41
  7. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 40
  8. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generationcited by 36
  9. 2025Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learningcited by 33
  10. 2024Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biasescited by 30
  11. 2025Directly Aligning the Full Diffusion Trajectory with Fine-Grained Human Preferencecited by 30
  12. 2024Flow Generator Matchingcited by 26
  13. 2024An End-to-End Robust Point Cloud Semantic Segmentation Network with Single-Step Conditional Diffusion Modelscited by 23
  14. 2025DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Modelscited by 23
  15. 2023AdaDiff: Adaptive Step Selection for Fast Diffusion Modelscited by 20
  16. 2024Constrained Diffusion Models via Dual Trainingcited by 20
  17. 2024Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learningcited by 19
  18. 2024Stochastic Control for Fine-tuning Diffusion Models: Optimality, Regularity, and Convergencecited by 19
  19. 2025Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generationcited by 19
  20. 2024Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrivecited by 18
  21. 2025Taming Preference Mode Collapse via Directional Decoupling Alignment in Diffusion Reinforcement Learningcited by 18
  22. 2023Beta Diffusioncited by 17
  23. 2024Towards Reliable Advertising Image Generation Using Human Feedbackcited by 16
  24. 2024Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Rewardcited by 16
  25. 2025FDPP: Fine-tune Diffusion Policy with Human Preferencecited by 15
  26. 2024Graph Diffusion Policy Optimizationcited by 13
  27. 2024AdaNAT: Exploring Adaptive Policy for Token-Based Image Generationcited by 13
  28. 2024Elucidating Optimal Reward-Diversity Tradeoffs in Text-to-Image Diffusion Modelscited by 13
  29. 2025Reinforcement Fine-Tuning for Materials Designcited by 13
  30. 2024Aligning Few-Step Diffusion Models with Dense Reward Difference Learningcited by 12
  31. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  32. 2026Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approachcited by 11
  33. 2025Composition and Alignment of Diffusion Models using Constrained Learningcited by 10
  34. 2025Seeing What Matters: Visual Preference Policy Optimization for Visual Generationcited by 10
  35. 2025Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generationcited by 9
  36. 2025Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedbackcited by 9
  37. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  38. 2025PrismAudio: Decomposed Chain-of-Thoughts and Multi-dimensional Rewards for Video-to-Audio Generationcited by 9

50 most cited of 81 All 81 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis
  2. 2025Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps
  3. 2022Accelerating Diffusion Models via Early Stop of the Diffusion Process
  4. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  5. 2025Pretrained Diffusion Models Are Inherently Skipped-Step Samplers
  6. 2025Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions
  7. 2025ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning
  8. 2025Understanding Sampler Stochasticity in Training Diffusion Models for RLHF