mlsanityData to 14 Oct 2026

ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization

arXiv 2406.04312 ↗First posted 6 Jun 2024cs.CVcited by 119

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Text-to-Image (T2I) models have made significant advancements in recent years, but they still struggle to accurately capture intricate details specified in complex compositional prompts. While fine-tuning T2I models with reward objectives has shown promise, it suffers from "reward hacking" and may not generalize well to unseen prompt distributions. In this work, we propose Reward-based Noise Optimization (ReNO), a novel approach that enhances T2I models at inference by optimizing the initial noise based on the signal from one or multiple human preference reward models. Remarkably, solving this optimization problem with gradient ascent for 50 iterations yields impressive results on four different one-step models across two competitive benchmarks, T2I-CompBench and GenEval. Within a computational budget of 20-50 seconds, ReNO-enhanced one-step models consistently surpass the performance of all current open-source Text-to-Image models. Extensive user studies demonstrate that our model is preferred nearly twice as often compared to the popular SDXL model and is on par with the proprietary Stable Diffusion 3 with 8B parameters. Moreover, given the same computational resources, a ReNO-optimized one-step model outperforms widely-used open-source models such as SDXL and PixArt-α, highlighting the efficiency and effectiveness of ReNO in enhancing T2I model performance at inference time. Code is available at https://github.com/ExplainableML/ReNO.

Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 84 of its 102 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

84 of 102 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation23 Apr
  2. 2024Aligning Diffusion Models by Optimizing Human Utility6 Apr
  3. 2024Social Reward: Evaluating and Enhancing Generative AI through Million-User Feedback from an Online Creative Community15 Feb
  4. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models13 Feb
  5. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov
  6. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  7. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep
  8. 2023Aligning Text-to-Image Models using Human Feedback23 Feb

Other references · 76

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  9. 2020Denoising Diffusion Implicit Modelscited by 9,781
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  12. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
38 more
  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2017Deep reinforcement learning from human preferencescited by 3,912
  11. 2018Neural Ordinary Differential Equationscited by 3,779
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  13. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  14. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  15. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  16. 2023Consistency Modelscited by 1,822
  17. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  18. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  19. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  20. 2021Variational Diffusion Modelscited by 1,153
  21. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  22. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  23. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  24. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  25. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  26. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  27. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  28. 2023Adversarial Diffusion Distillationcited by 759
  29. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  30. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  31. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  32. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  33. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  34. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
  35. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  36. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  37. 2023Universal Guidance for Diffusion Modelscited by 437
  38. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 54; 2026: 55.

119 papers.

Sort:

By area:

  1. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  2. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  3. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
  4. 2024A Noise is Worth Diffusion Guidancecited by 49
  5. 2025Scaling Image and Video Generation via Test-Time Evolutionary Searchcited by 40
  6. 2025Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcingcited by 38
  7. 2025Diffusion Model as a Noise-Aware Latent Reward Model for Step-Level Preference Optimizationcited by 37
  8. 2024Alignment of Diffusion Models: Fundamentals, Challenges, and Futurecited by 35
  9. 2025Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Modelscited by 35
  10. 2024Scalable Ranked Preference Optimization for Text-to-Image Generationcited by 33
  11. 2026Diamond Maps: Efficient Reward Alignment via Stochastic Flow Mapscited by 29
  12. 2024DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Schedulingcited by 27
38 more
  1. 2025Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generationcited by 19
  2. 2025Joint Distillation for Fast Likelihood Evaluation and Sampling in Flow-based Modelscited by 18
  3. 2025GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Modelscited by 17
  4. 2025Test-time scaling of diffusions with flow mapscited by 16
  5. 2026Action-to-Action Flow Matchingcited by 16
  6. 2025Science-T2I: Addressing Scientific Illusions in Image Synthesiscited by 15
  7. 2024The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generationcited by 14
  8. 2024Elucidating Optimal Reward-Diversity Tradeoffs in Text-to-Image Diffusion Modelscited by 13
  9. 2025Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Predictioncited by 12
  10. 2025Dual-Process Image Generationcited by 12
  11. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 12
  12. 2024ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generationcited by 11
  13. 2024TKG-DM: Training-free Chroma Key Content Generation Diffusion Modelcited by 11
  14. 2025It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Modelscited by 11
  15. 2025How far can we go with ImageNet for Text-to-Image generation?cited by 10
  16. 2025ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generationcited by 10
  17. 2025Model Already Knows the Best Noise: Bayesian Active Noise Selection via Attention in Video Diffusion Modelcited by 10
  18. 2025SuperFlow: Training Flow Matching Models with RL on the Flycited by 10
  19. 2024Attention Overlap Is Responsible for The Entity Missing Problem in Text-to-image Diffusion Models!cited by 9
  20. 2025MIRA: Towards Mitigating Reward Hacking in Inference-Time Alignment of T2I Diffusion Modelscited by 9
  21. 2025MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Trainingcited by 8
  22. 2025Fast Direct: Query-Efficient Online Black-box Guidance for Diffusion-model Target Generationcited by 7
  23. 2025Towards Better Optimization For Listwise Preference in Diffusion Modelscited by 7
  24. 2025Adjusting Initial Noise to Mitigate Memorization in Text-to-Image Diffusion Modelscited by 7
  25. 2025Inversion-DPO: Precise and Efficient Post-Training for Diffusion Modelscited by 6
  26. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
  27. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 6
  28. 2026How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidancecited by 6
  29. 2025Learning User Preferences for Image Generation Modelcited by 5
  30. 2024FreeCond: Free Lunch in the Input Conditions of Text-Guided Inpaintingcited by 4
  31. 2025Fine-Grained Alignment and Noise Refinement for Compositional Text-to-Image Generationcited by 4
  32. 2025Controlling Latent Diffusion Using Latent CLIPcited by 4
  33. 2025Straighten Viscous Rectified Flow via Noise Optimizationcited by 4
  34. 2025Evaluating the Evaluators: Metrics for Compositional Text-to-Image Generationcited by 4
  35. 2025SONIC: Spectral Optimization of Noise for Inpainting with Consistencycited by 4
  36. 2025AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Pathcited by 4
  37. 2026HyperAlign: Hypernetwork for Efficient Test-Time Alignment of Diffusion Modelscited by 4
  38. 2026Few-Step Diffusion Sampling Through Instance-Aware Discretizationscited by 4

50 most cited of 119 All 119 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models
  2. 2025CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration
  3. 2025PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting
  4. 2026DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation
  5. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  6. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  7. 2026TILT: Improving Compositional Generation in Diffusion Models with a Model-Intrinsic Reward
  8. 2024Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models