mlsanityData to 14 Oct 2026

A General Framework for Inference-time Scaling and Steering of Diffusion Models

arXiv 2501.06848 ↗First posted 12 Jan 2025cs.LG cs.CL cs.CVcited by 224

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

Diffusion models produce impressive results in modalities ranging from images and video to protein design and text. However, generating samples with user-specified properties remains a challenge. Recent research proposes fine-tuning models to maximize rewards that capture desired properties, but these methods require expensive training and are prone to mode collapse. In this work, we present Feynman-Kac (FK) steering, an inference-time framework for steering diffusion models with reward functions. FK steering works by sampling a system of multiple interacting diffusion processes, called particles, and resampling particles at intermediate steps based on scores computed using functions called potentials. Potentials are defined using rewards for intermediate states and are selected such that a high value indicates that the particle will yield a high-reward sample. We explore various choices of potentials, intermediate rewards, and samplers. We evaluate FK steering on text-to-image and text diffusion models. For steering text-to-image models with a human preference reward, we find that FK steering a 0.8B parameter model outperforms a 2.6B parameter fine-tuned model on prompt fidelity, with faster sampling and no training. For steering text diffusion models with rewards for text quality and specific text attributes, we find that FK steering generates lower perplexity, more linguistically acceptable outputs and enables gradient-free control of attributes like toxicity. Our results demonstrate that inference-time scaling and steering of diffusion models - even with off-the-shelf rewards - can provide significant sample quality gains and controllability benefits. Code is available at https://github.com/zacharyhorvitz/Fk-Diffusion-Steering .

Key facts. Cited by 224 papers in the map (data through 14 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 29 of its 69 references that are in the map. First posted on arXiv 12 Jan 2025.

Builds on

29 of 69 references.

References · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  9. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  10. 2021Variational Diffusion Modelscited by 1,153
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2023Training Diffusion Models with Reinforcement Learningcited by 982
17 more
  1. 2018Neural Network Acceptability Judgmentscited by 958
  2. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  3. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  5. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 236
  6. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 233
  7. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  8. 2023Protein Design with Guided Discrete Diffusioncited by 157
  9. 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
  10. 2023Likelihood-Based Diffusion Language Modelscited by 151
  11. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
  12. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  13. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  14. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  15. 2023Monte Carlo guided Diffusion for Bayesian linear inverse problemscited by 66
  16. 2024Conditional sampling within generative diffusion modelscited by 19
  17. 2023ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfercited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 98; 2026: 126.

224 papers.

Sort:

By area:

  1. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  2. 2025Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Expertscited by 96
  3. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88
  4. 2025From Reflection to Perfection: Scaling Inference-Time Optimization for Text-to-Image Diffusion Models via Reflection Tuningcited by 56
  5. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 49
  6. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  7. 2026Categorical Flow Mapscited by 44
  8. 2025Reflect-DiT: Inference-Time Scaling for Text-to-Image Diffusion Transformers via In-Context Reflectioncited by 43
  9. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 41
  10. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 40
  11. 2025Fine-Tuning Discrete Diffusion Models with Policy Gradient Methodscited by 38
  12. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 37
38 more
  1. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 37
  2. 2026Q-learning with Adjoint Matchingcited by 37
  3. 2026Meta Flow Maps enable scalable reward alignmentcited by 37
  4. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 34
  5. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 31
  6. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Mapscited by 29
  7. 2026Discrete Flow Mapscited by 29
  8. 2025Scaling Offline RL via Efficient and Expressive Shortcut Modelscited by 27
  9. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimationcited by 27
  10. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  11. 2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 24
  12. 2025DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Modelscited by 23
  13. 2025Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphscited by 21
  14. 2025Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlocited by 20
  15. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  16. 2025Split Gibbs Discrete Diffusion Posterior Samplingcited by 18
  17. 2025RNE: plug-and-play diffusion inference-time control and energy-based trainingcited by 17
  18. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Modelscited by 17
  19. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Modelscited by 17
  20. 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generationcited by 17
  21. 2025Diffusion models for inverse problemscited by 16
  22. 2025Test-time scaling of diffusions with flow mapscited by 16
  23. 2025SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steeringcited by 15
  24. 2025TTS-VAR: A Test-Time Scaling Framework for Visual Auto-Regressive Generationcited by 15
  25. 2026VLS: Steering Pretrained Robot Policies via Vision-Language Modelscited by 15
  26. 2025LinEAS: End-to-end Learning of Activation Steering with a Distributional Losscited by 14
  27. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Designcited by 14
  28. 2025TR2-D2: Tree Search Guided Trajectory-Aware Fine-Tuning for Discrete Diffusioncited by 14
  29. 2025Flow Map Distillation Without Datacited by 14
  30. 2025Value Gradient Guidance for Flow Matching Alignmentcited by 14
  31. 2025Dynamic Classifier-Free Diffusion Guidance via Online Feedbackcited by 13
  32. 2026Enhanced Diffusion Sampling: Efficient Rare Event Sampling and Free Energy Calculation with Diffusion Modelscited by 13
  33. 2025ScalingNoise: Scaling Inference-Time Search for Generating Infinite Videoscited by 12
  34. 2025Dual-Process Image Generationcited by 12
  35. 2025Navigating with Annealing Guidance Scale in Diffusion Spacecited by 11
  36. 2025Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Designcited by 11
  37. 2025CREPE: Controlling Diffusion with Replica Exchangecited by 11
  38. 2026Discrete Feynman-Kac Correctorscited by 11

50 most cited of 224 All 224 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models
  2. 2026Learning Sampling Parameters for Diffusion Models
  3. 2026Error-Corrected Inference-Time Scaling for Imperfect Diffusion Models
  4. 2024Bigger is not Always Better: Scaling Properties of Latent Diffusion Models
  5. 2026Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning
  6. 2026Specificity-Aware Diffusion Steering via Variance-Reduced Sequential Monte Carlo
  7. 2024Plug-and-Play Diffusion Distillation
  8. 2026ATHENA: Adaptive Test-Time Steering for Improving Count Fidelity in Diffusion Models