DOHF: Online Diffusion Fine-tuning with Doob's h-transform Guidance
Not on the map. Why?
Abstract (arXiv)
Reward-based diffusion fine-tuning faces practical challenges when desirable outcomes are rare or conditioning corrections are costly to estimate. In this work, we propose Diffusion Online h-guidance Fine-tuning (DOHF), which turns Doob's h-transform into a practical online training algorithm. DOHF assigns optimality weights to generated samples, estimates the normalized local correction nablalog h under the current rollout policy, and distills it directly into the generative model. Theoretically, we characterize the population-optimal DiffusionNFT update as well as the various classfier free guidance methods through a unified h-transform perspective. Methodologically, our framework accommodates black-box and non-differentiable rewards without additional network evaluations. We further show improved alignments under three empirical scenarios. Our work demonstrates how adapting probabilistic conditioning through inexpensive estimation and iterative distillation can improve generative learning across statistical sampling and visual generation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
32 of 49 references.
References · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
- 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 233
- 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 232
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
- 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
- 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 183
20 more
- 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
- 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 98
- 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 67
- 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
- 2024DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised h-transformcited by 46
- 2025Online Reward-Weighted Fine-Tuning of Flow Matching with Wasserstein Regularizationcited by 43
- 2026Meta Flow Maps enable scalable reward alignmentcited by 37
- 2025Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learningcited by 33
- 2024Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Surveycited by 23
- 2024Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learningcited by 19
- 2026Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approachcited by 11
- 2026Training-Free Adaptation of Diffusion Models via Doob's h-Transformcited by 11
- 2025Understanding Sampler Stochasticity in Training Diffusion Models for RLHFcited by 7
- 2026Reward Score Matching: Unifying Reward-based Fine-tuning for Flow and Diffusion Modelscited by 7
- 2026FlowAWR: Online Adaptive Flow Reinforcement via Advantage-Weighted Rectificationcited by 6
- 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matchingcited by 5
- 2026Improved techniques for fine-tuning flow models via adjoint matching: a deterministic control pipelinecited by 5
- 2025Iterative Tilting for Diffusion Fine-Tuningcited by 4
- 2026Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space Viewcited by 3
- 2026Scaling Reinforcement Learning for Diffusion Models via Velocity Matchingcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.