mlsanityData to 14 Oct 2026

AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment

arXiv 2605.17602 ↗First posted 17 May 2026cs.AI cs.CV cs.LGcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Aligning Text-to-Image (T2I) generation models with human preferences increasingly relies on image reward models that score or rank generated images according to prompt alignment and perceptual quality. Existing reward models are commonly trained as Bradley-Terry (BT) preference models on large-scale human preference corpora, making them costly to train, difficult to adapt, and opaque in their evaluation criteria. Meanwhile, Vision-Language Model (VLM) judges can provide more fine-grained assessments through textual rubrics, but their manually designed or heuristically generated scoring rules may fail to reliably reflect human preferences. In this paper, we propose AutoRubric-T2I, the first rubric learning framework in T2I that automatically synthesizes and selects explicit rubrics for guiding VLM judges. AutoRubric-T2I first synthesizes reasoning traces from preference pairs into candidate rubrics, then uses a VLM judge to score paired images under each rubric, producing pairwise rubric-score differences for preference learning. To remove noisy and redundant rules, we further employ a ℓ1-Regularized Logistic Regression Refiner, which selects the Top-N most discriminative rubrics. Extensive evaluations show that AutoRubric-T2I produces high-quality, interpretable reward signals using less than 0.01% of the annotated preference data, substantially reducing the need for large-scale reward-model training. On image reward benchmarks such as MMRB2, AutoRubric-T2I outperforms strong reward model baselines. We further validate AutoRubric-T2I as an RL reward on downstream T2I tasks, including TIIF and UniGenBench++, where it improves generation quality over scalar reward models using the Flow-GRPO pipeline on diffusion models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 29 of its 38 references that are in the map. First posted on arXiv 17 May 2026.

Builds on

29 of 38 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Understanding Reward Hacking in Text-to-Image Reinforcement Learning6 Jan
  2. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image18 Dec
  3. 2025RubricRL: Simple Generalizable Rewards for Text-to-Image Generation25 Nov
  4. 2025HPSv3: Towards Wide-Spectrum Human Preference Score5 Aug
  5. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  6. 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
  7. 2024Automated Filtering of Human Feedback Data for Aligning Text-to-Image Diffusion Models14 Oct

Other references · 22

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  5. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  6. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  7. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  8. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  9. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  10. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  11. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  12. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 136
10 more
  1. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
  2. 2025Reinforcement Learning with Rubric Anchorscited by 113
  3. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 82
  4. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 69
  5. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 54
  6. 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Trainingcited by 53
  7. 2026RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generationcited by 52
  8. 2025Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modelingcited by 30
  9. 2025UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generationcited by 25
  10. 2025AutoRule: Reasoning Chain-of-thought Extracted Rule-based Rewards Improve Preference Learningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025InstructEngine: Instruction-driven Text-to-Image Alignment
  2. 2025Free Lunch Alignment of Text-to-Image Diffusion Models without Preference Image Pairs
  3. 2025YINYANG-ALIGN: Benchmarking Contradictory Objectives and Proposing Multi-Objective Optimization based DPO for Text-to-Image Alignment
  4. 2025SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner
  5. 2024Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Models
  6. 2023Filter & Align: Leveraging Human Knowledge to Curate Image-Text Data
  7. 2025Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences
  8. 2024Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation