mlsanityData to 7 Oct 2026

Dynamic Search for Inference-Time Alignment in Diffusion Models

arXiv 2503.02039 ↗First posted 3 Mar 2025cs.LG cs.AIcited by 36

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have shown promising generative capabilities across diverse domains, yet aligning their outputs with desired reward functions remains a challenge, particularly in cases where reward functions are non-differentiable. Some gradient-free guidance methods have been developed, but they often struggle to achieve optimal inference-time alignment. In this work, we newly frame inference-time alignment in diffusion as a search problem and propose Dynamic Search for Diffusion (DSearch), which subsamples from denoising processes and approximates intermediate node rewards. It also dynamically adjusts beam width and tree expansion to efficiently explore high-reward generations. To refine intermediate decisions, DSearch incorporates adaptive scheduling based on noise levels and a lookahead heuristic function. We validate DSearch across multiple domains, including biological sequence design, molecular optimization, and image generation, demonstrating superior reward optimization compared to existing approaches.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 68 of its 101 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

68 of 101 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan
  2. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Models12 Jan
  3. 2025Test-time Alignment of Diffusion Models without Reward Over-optimization10 Jan
  4. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding15 Aug
  5. 2024Gradient Guidance for Diffusion Models: An Optimization Perspective23 Apr
  6. 2024Particle Denoising Diffusion Sampler9 Feb
  7. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model17 Mar
  8. 2023Universal Guidance for Diffusion Models14 Feb
  9. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  12. 2020Denoising Diffusion Implicit Models6 Oct
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2018How Powerful are Graph Neural Networks?cited by 4,355
  7. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  12. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
38 more
  1. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  2. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  3. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  5. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  6. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  7. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  8. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  9. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
  10. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  11. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  12. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  13. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  14. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  15. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  16. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  17. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 291
  18. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 273
  19. 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 239
  20. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  21. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  22. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  23. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 160
  24. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  25. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 156
  26. 2023Controlled Decoding from Language Modelscited by 133
  27. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  28. 2024Theoretical guarantees on the best-of-n alignment policycited by 116
  29. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  30. 2022Exploring Chemical Space with Score-based Out-of-distribution Generationcited by 95
  31. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  32. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 85
  33. 2021Learning and Planning in Complex Action Spacescited by 76
  34. 2021Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generationcited by 68
  35. 2023Monte Carlo guided Diffusion for Bayesian linear inverse problemscited by 65
  36. 2020Monte-Carlo Tree Search as Regularized Policy Optimizationcited by 59
  37. 2021Machine Translation Decoding beyond Beam Searchcited by 55
  38. 2024Value Augmented Sampling for Language Model Alignment and Personalizationcited by 49

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 17.

36 papers.

Sort:

By area:

  1. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 38
  2. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 32
  3. 2026Meta Flow Maps enable scalable reward alignmentcited by 32
  4. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  5. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Mapscited by 26
  6. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Modelscited by 17
  7. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  8. 2025Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Designcited by 10
  9. 2025ImagerySearch: Adaptive Test-Time Search for Video Generation Beyond Semantic Dependency Constraintscited by 9
  10. 2025Adaptive Inference-Time Scaling via Cyclic Diffusion Searchcited by 7
  11. 2025Fast Monte Carlo Tree Diffusion: 100x Speedup via Parallel Sparse Planningcited by 6
  12. 2026Training-Free Adaptation of Diffusion Models via Doob's h-Transformcited by 6
24 more
  1. 2026Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generationcited by 6
  2. 2025Tree Reward-Aligned Search for TReASURe in Masked Diffusion Language Modelscited by 2
  3. 2025Diffusion Alignment as Variational Expectation-Maximizationcited by 2
  4. 2026Stitched Value Model for Diffusion Alignmentcited by 2
  5. 2026A2D2: Fine-Tuning Any-Length Discrete Diffusion for Adaptive Decodingcited by 2
  6. 2026Flash-BoN: Instant Drafts for Inference-Time Scaling in Diffusion Modelscited by 2
  7. 2025Inference-Time Alignment of Diffusion Models via Evolutionary Algorithmscited by 1
  8. 2025Personalized Constitutionally-Aligned Agentic Superego: Secure AI Behavior Aligned to Diverse Human Valuescited by 1
  9. 2025Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorationscited by 1
  10. 2025Language Models for Controllable DNA Sequence Designcited by 1
  11. 2025Inference-Time Search Using Side Information for Diffusion-Based Image Reconstructioncited by 1
  12. 2025Rethinking Test Time Scaling for Flow-Matching Generative Modelscited by 1
  13. 2026Tuning Real-World Image Restoration at Inference: A Test-Time Scaling Paradigm for Flow Matching Modelscited by 1
  14. 2026NoiseRater: Meta-Learned Noise Valuation for Diffusion Model Trainingcited by 1
  15. 2026BFMT: Enhancing Search Capabilities of Tree Sampler via Bootstrap Flow-Map Treecited by 1
  16. 2026Inference-Time Scaling of Diffusion Models via Progressive Seed Pruningcited by 1
  17. 2025Seek and You Shall Foldcited by 0
  18. 2025Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisationcited by 0
  19. 2025Iterative Inference-time Scaling with Adaptive Frequency Steering for Image Super-Resolutioncited by 0
  20. 2026Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Modelscited by 0
  21. 2026VASR: Variance-Aware Systematic Resampling for Reward-Guided Diffusioncited by 0
  22. 2026Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Fieldcited by 0
  23. 2026Test-Time Scaling in Multimodal Foundation Models: A Comprehensive Survey of Generation and Reasoningcited by 0
  24. 2026On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review
  2. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimization
  3. 2025Inference-time Scaling of Diffusion Models through Classical Search
  4. 2025Unified Control for Inference-Time Guidance of Denoising Diffusion Models
  5. 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching
  6. 2026Spectral Feedback for Test-Time Alignment of Protein Diffusion Models
  7. 2025Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models
  8. 2025MIRA: Towards Mitigating Reward Hacking in Inference-Time Alignment of T2I Diffusion Models