mlsanityData to 7 Oct 2026

DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation

arXiv 2605.25876 ↗First posted 25 May 2026cs.CVcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

With the continued advancement of text-to-image (T2I) generation, producing high-quality images is becoming increasingly attainable; consequently, user demands are shifting toward images that better satisfy their specific requirements. As reward models play an increasingly important role in assessing whether generated images align with user preference, this trend introduces an important challenge for reward modeling: rather than relying solely on static and general evaluation dimensions, reward models should account for the task-relevant and fine-grained criteria through which users assess whether generated images meet their specific requirements. To address this challenge, we propose DyCoRM, a dynamic, criterion-aware reward model that grounds task-relevant criteria and performs criterion-aware preference comparison. To support this setting, we construct DyCoDataset-20K, which provides dynamic criteria together with criterion-level annotations, and further derive DyCoBench-1K, a benchmark for systematically evaluating reward models under dynamic criteria. We further introduce DyCoPick, which applies criterion-aware reward modeling to selecting T2I images. Our contributions establish the first reward modeling framework for dynamic and fine-grained evaluation and practical application in T2I generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 44 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

39 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Aligning Diffusion Models by Optimizing Human Utility6 Apr
  2. 2024Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation11 Jan
  3. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  4. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  5. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference25 Mar
  6. 2023Aligning Text-to-Image Models using Human Feedback23 Feb

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  8. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2025Qwen-Image Technical Reportcited by 973
  12. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
21 more
  1. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  2. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  3. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  4. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  5. 2016Generative Adversarial Text to Image Synthesiscited by 735
  6. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  7. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  8. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  9. 2024MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmarkcited by 362
  10. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  11. 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
  12. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  13. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
  14. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
  15. 2025Qwen2.5-1M Technical Reportcited by 158
  16. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
  17. 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 105
  18. 2024Learning Multi-dimensional Human Preference for Text-to-Image Generationcited by 102
  19. 2025GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluationcited by 42
  20. 2022Human Evaluation of Text-to-Image Models on a Multi-Task Benchmarkcited by 41
  21. 2024EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluationcited by 37

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Personalized Reward Modeling for Text-to-Image Generation
  2. 2025Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models
  3. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  4. 2026Traversing the Satisfaction-Diversity Frontier in Text-to-Image Diffusion
  5. 2026DiT-Reward: Generative Representations for Text-to-Image Reward Modeling
  6. 2024Beyond Thumbs Up/Down: Untangling Challenges of Fine-Grained Feedback for Text-to-Image Generation
  7. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization
  8. 2025RubricRL: Simple Generalizable Rewards for Text-to-Image Generation