Iterative Tilting for Diffusion Fine-Tuning
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
We introduce iterative tilting, a gradient-free method for fine-tuning diffusion models toward reward-tilted distributions. The method decomposes a large reward tilt exp(λ r) into N sequential smaller tilts, each admitting a tractable score update via first-order Taylor expansion. This requires only forward evaluations of the reward function and avoids backpropagating through sampling chains. We validate on a two-dimensional Gaussian mixture with linear reward, where the exact tilted distribution is available in closed form.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 20 of its 24 references that are in the map. First posted on arXiv 2 Dec 2025.
Builds on
20 of 24 references.
Closest ancestors · 8
Same direction, newest first
- 2024Fine-tuning of diffusion models via stochastic control: entropy regularization and beyond10 Mar
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesis5 Mar
- 2022Classifier-Free Diffusion Guidance26 Jul
- 2021Diffusion Models Beat GANs on Image Synthesis11 May
- 2021Improved Denoising Diffusion Probabilistic Models18 Feb
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
- 2020Denoising Diffusion Implicit Models6 Oct
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 12
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
- 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 233
- 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 143
- 2023Stochastic Optimal Control Matchingcited by 40
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matchingcited by 5
- 2026Diffusion Reinforcement Learning via Centered Reward Distillationcited by 5
- 2026DOHF: Online Diffusion Fine-tuning with Doob's h-transform Guidancecited by 1
- 2026Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusioncited by 0
- 2026DOHF: Online Diffusion Fine-tuning with Doob's h-transform Guidance25 Sep 2026
- 2026Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion30 Aug 2026
- 2026Diffusion Reinforcement Learning via Centered Reward Distillation14 Mar 2026
- 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching10 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2026A unified perspective on fine-tuning and sampling with diffusion and flow models
- 2026Steering diffusion models with quadratic rewards: a fine-grained analysis
- 2025Tilt Matching for Scalable Sampling and Fine-Tuning
- 2025Test-time Alignment of Diffusion Models without Reward Over-optimization
- 2026Are we really tilting? The mechanics of reward guidance in flow and diffusion models
- 2025PIRF: Physics-Informed Reward Fine-Tuning for Diffusion Models
- 2025Fine-Tuning Diffusion Models via Intermediate Distribution Shaping
- 2025Effective Test-Time Scaling of Discrete Diffusion through Iterative Refinement