mlsanityData to 7 Oct 2026

Trust-Region Noise Search for Black-Box Alignment of Diffusion and Flow Models

arXiv 2603.14504 ↗First posted 15 Mar 2026cs.LG cs.AI cs.CVcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Optimizing the noise samples of diffusion and flow models is an increasingly popular approach to align these models to target rewards at inference time. However, we observe that these approaches are usually restricted to differentiable or cheap reward models, the formulation of the underlying pretrained generative model, or are memory/compute inefficient. We instead propose a simple trust-region based search algorithm (TRS) which treats the pre-trained generative and reward models as a black-box and only optimizes the source noise. Our approach achieves a good balance between global exploration and local exploitation, and is versatile and easily adaptable to various generative settings and reward models with minimal hyperparameter tuning. We evaluate TRS across text-to-image, molecule and protein design tasks, and obtain significantly improved output samples over the base generative models and other inference-time alignment approaches which optimize the source noise sample, or even the entire reverse-time sampling noise trajectories in the case of diffusion models. Our source code is publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 31 of its 48 references that are in the map. First posted on arXiv 15 Mar 2026.

Builds on

31 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Test-time Alignment of Diffusion Models without Reward Over-optimization10 Jan
  2. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization6 Jun
  3. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep

Other references · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  12. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
16 more
  1. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  2. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  3. 2023Universal Guidance for Diffusion Modelscited by 426
  4. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  5. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 201
  6. 20233D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Predictioncited by 176
  7. 2023SE(3)-Stochastic Flow Matching for Protein Backbone Generationcited by 147
  8. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  9. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
  10. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 115
  11. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
  12. 2024Inference-Time Alignment of Diffusion Models with Direct Noise Optimizationcited by 51
  13. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 36
  14. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  15. 2024Training Free Guided Flow Matching with Optimal Controlcited by 23
  16. 2025Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generationcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Gradient-Free Noise Optimization for Reward Alignment in Generative Models
  2. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review
  3. 2025Inference-Time Alignment of Diffusion Models via Evolutionary Algorithms
  4. 2025Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning
  5. 2026NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment
  6. 2026PG-MAP: Joint MAP Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models
  7. 2025Disentangling Total-Variance and Signal-to-Noise-Ratio Improves Diffusion Models
  8. 2026TRACE: Transport Alignment Conformal Prediction via Diffusion and Flow Matching Models