mlsanityData to 7 Oct 2026

Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design

arXiv 2502.14944 ↗First posted 20 Feb 2025cs.LG cs.AI cs.NEcited by 21

On the map: Diffusion models → Flow matching, time series and science

Abstract (arXiv)

To fully leverage the capabilities of diffusion models, we are often interested in optimizing downstream reward functions during inference. While numerous algorithms for reward-guided generation have been recently proposed due to their significance, current approaches predominantly focus on single-shot generation, transitioning from fully noised to denoised states. We propose a novel framework for inference-time reward optimization with diffusion models inspired by evolutionary algorithms. Our approach employs an iterative refinement process consisting of two steps in each iteration: noising and reward-guided denoising. This sequential refinement allows for the gradual correction of errors introduced during reward optimization. Besides, we provide a theoretical guarantee for our framework. Finally, we demonstrate its superior empirical performance in protein and cell-type-specific regulatory DNA design. The code is available at hrefhttps://github.com/masa-ue/ProDifEvo-Refinementhttps://github.com/masa-ue/ProDifEvo-Refinement.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Placed in Flow matching, time series and science, one of 14 directions in Diffusion models. Builds on 34 of its 58 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

34 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design17 Oct
  2. 2024Dirichlet Flow Matching with Applications to DNA Sequence Design8 Feb
  3. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Models30 Jun

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  8. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  9. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  11. 2024Simple and Effective Masked Diffusion Language Modelscited by 800
  12. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
19 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  2. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  3. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  4. 2023Universal Guidance for Diffusion Modelscited by 426
  5. 2022Generating Sequences by Learning to Self-Correctcited by 271
  6. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  7. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  8. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
  9. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  10. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 124
  11. 2017Generating Sentences by Editing Prototypescited by 88
  12. 2023Monte Carlo guided Diffusion for Bayesian linear inverse problemscited by 65
  13. 2024Particle Denoising Diffusion Samplercited by 63
  14. 2019Elements of Sequential Monte Carlocited by 60
  15. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 59
  16. 2024Feedback Efficient Online Fine-Tuning of Diffusion Modelscited by 52
  17. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 8
  18. 2023Extrapolative Controlled Sequence Generation via Iterative Refinementcited by 7
  19. 2016Iterative Refinement for Machine Translationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 12; 2026: 9.

21 papers.

Sort:

By area:

  1. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  2. 2025SteerX: Creating Any Camera-Free 3D and 4D Scenes with Geometric Steeringcited by 13
  3. 2025Inference-Time Scaling of Discrete Diffusion Models via Importance Weighting and Optimal Proposal Designcited by 12
  4. 2025Autonomous Agents for Scientific Discovery: Orchestrating Scientists, Language, Code, and Physicscited by 11
  5. 2026CORE: Context-Robust Remasking for Diffusion Language Modelscited by 10
  6. 2025It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Modelscited by 9
  7. 2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
  8. 2025MIRA: Towards Mitigating Reward Hacking in Inference-Time Alignment of T2I Diffusion Modelscited by 7
  9. 2025Posterior Inference in Latent Space for Scalable Constrained Black-box Optimizationcited by 4
  10. 2025VARD: Efficient and Dense Fine-Tuning for Diffusion Models with Value-based RLcited by 3
  11. 2025Ctrl-DNA: Controllable Cell-Type-Specific Regulatory DNA Design via Constrained RLcited by 3
  12. 2025ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoodscited by 3
9 more
  1. 2026Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matchingcited by 2
  2. 2025Language Models for Controllable DNA Sequence Designcited by 1
  3. 2026LENS: Low-Frequency Eigen Noise Shaping for Efficient Diffusion Samplingcited by 1
  4. 2026From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactionscited by 0
  5. 2026LPDP: Inference-Time Reward Control for Variable-Length DNA Generation with Edit Flowscited by 0
  6. 2026Generative Refinement for Low-Budget Black-Box Optimizationcited by 0
  7. 2026On the Design Space of Discrete Diffusion Online Adaptation for Molecular Optimizationcited by 0
  8. 2026When Genomic Masking Priors Fail to Transfer: Strong Variant Prediction, Weak Functional Generationcited by 0
  9. 2026Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review
  2. 2025Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design
  3. 2026Reward-Guided Discrete Diffusion via Clean-Sample Markov Chain for Molecule and Biological Sequence Design
  4. 2024Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models
  5. 2026VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion
  6. 2025Dynamic Search for Inference-Time Alignment in Diffusion Models
  7. 2026IDRF: Inverse-Distilled Reward Fine-tuning of Masked Discrete Diffusion Models
  8. 2023Protein Design with Guided Discrete Diffusion