mlsanityData to 14 Oct 2026

MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and efficiency

arXiv 2510.25897 ↗First posted 29 Oct 2025cs.CV cs.LGcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

The default paradigm of post-training text-to-image generators includes post-hoc selection of generated images, and subsequent training with one reward model to align the generator to the reward, typically user preference. This discards informative data as well as optimizes only for a single reward, hence harming diversity, semantic fidelity and efficiency. Instead, we propose MIRO, a method that conditions the model on multiple rewards during training, thus letting the model learn user preferences directly. MIRO pre-training both improves the visual quality of the generated images and speeds up the training, achieving state of the art on the GenEval compositional benchmark and user-preference scores (PickAScore, ImageReward, HPSv2).

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 72 of its 97 references that are in the map. First posted on arXiv 29 Oct 2025.

Builds on

72 of 97 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generation17 Mar
  2. 2024Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Reward22 Nov
  3. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization6 Jun
  4. 2024Aligning Diffusion Models by Optimizing Human Utility6 Apr
  5. 2024RL for Consistency Models: Faster Reward Guided Text-to-Image Generation25 Mar
  6. 2024PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models13 Feb
  7. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  8. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep

Other references · 64

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  9. 2022Scalable Diffusion Models with Transformerscited by 6,054
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2022Flow Matching for Generative Modelingcited by 5,702
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2023Consistency Modelscited by 1,822
  7. 2020GLU Variants Improve Transformercited by 1,804
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2019Root Mean Square Layer Normalizationcited by 1,480
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  11. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  12. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  13. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  14. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  15. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  16. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  17. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  18. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  19. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  20. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  21. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  22. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  23. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  24. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  25. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  26. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 299
  27. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
  28. 2020Query-Key Normalization for Transformerscited by 280
  29. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 243
  30. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  31. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 233
  32. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  33. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 175
  34. 2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 143
  35. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  36. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 122
  37. 2019Training Agents using Upside-Down Reinforcement Learningcited by 107
  38. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88

50 most cited of 64 All 64 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Surflo: Consistent 3D Surface Flow Model with Global Statecited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  2. 2026PromptEcho: Annotation-Free Reward from Vision-Language Models for Text-to-Image Reinforcement Learning
  3. 2026DiT-Reward: Generative Representations for Text-to-Image Reward Modeling
  4. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation
  5. 2026Traversing the Satisfaction-Diversity Frontier in Text-to-Image Diffusion
  6. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  7. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
  8. 2026Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation