RubricRM: Generative Reward Modeling via Dynamic Rubrics for Image Generation and Editing
On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
Abstract (arXiv)
Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 19 of its 36 references that are in the map. First posted on arXiv 27 Aug 2026.
Builds on
19 of 36 references.
Closest ancestors · 8
Same direction, newest first
- 2026AutoRubric-T2I: Robust Rule-Based Reward Model for Text-to-Image Alignment17 May
- 2026Unified Personalized Reward Model for Vision Generation2 Feb
- 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editing30 Sep
- 2025EditScore: Unlocking Online RL for Image Editing via High-Fidelity Reward Modeling28 Sep
- 2025HPSv3: Towards Wide-Spectrum Human Preference Score5 Aug
- 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning6 May
- 2024VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation30 Dec
- 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preference25 Mar
Other references · 11
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 196
- 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
- 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
- 2026RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generationcited by 42
- 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteriacited by 8
- 2026Rationale Matters: Learning Transferable Rubrics via Proxy-Guided Critique for VLM Reward Modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025RubricRL: Simple Generalizable Rewards for Text-to-Image Generation
- 2026Think Before You Score: Thinking Reward Model for Visual Generation
- 2026Edit-Compass & EditReward-Compass: A Unified Benchmark for Image Editing and Reward Modeling
- 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
- 2026Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
- 2026DyCoRM: Dynamic Criterion-Aware Reward Modeling for Text-to-Image Generation
- 2026RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
- 2026Leveraging Verifier-Based Reinforcement Learning in Image Editing