Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models also fits: VLM benchmarks and visual reasoning
also in 1 more
Abstract (arXiv)
Precise instruction following in image generation, such as satisfying object counts and spatial relations, remains an open challenge at least in part because it is learned using unreliable reward models such as object detectors and vision-language models. We introduce Verifiable Visual Rewards (VVR), the first framework for programmatically verifiable image rewards, and show that training on it generalizes to natural prompts. Each VVR task is a scene of geometric objects and relations among them, from which we derive both the prompt and a deterministic verifier, so tasks can be generated in any number and at any chosen complexity. We release VVRBench, with 10,000 tasks over 32 constraint types, and VVRBench-Challenge, with 720 more complex tasks; the strongest model we evaluate---GPT-Image-2.5---solves 21.4% of VVRBench-Challenge. Using VVR scores as rewards for reinforcement learning (RLVVR) raises the accuracy of Stable Diffusion 3.5 Medium on VVRBench from 2.8% to 28.3% and demonstrates consistent easy-to-hard generalization. These gains extend to out-of-domain benchmarks, and mixing VVR into existing objectives further improves overall performance and human preference, motivating the adoption of VVR into standard image generation post-training recipes.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 60 references that are in the map. First posted on arXiv 28 Sep 2026.
Builds on
45 of 60 references.
References · 45
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
- 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
33 more
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 431
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
- 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 196
- 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
- 2025Generalizing Verifiable Instruction Followingcited by 178
- 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
- 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
- 2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 64
- 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficultycited by 60
- 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
- 2025SynLogic: Synthesizing Verifiable Reasoning Data at Scale for Learning Logical Reasoning and Beyondcited by 47
- 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
- 2025GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluationcited by 42
- 2025Rendering-Aware Reinforcement Learning for Vector Graphics Generationcited by 36
- 2024Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biasescited by 29
- 2024Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)cited by 17
- 2023Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generationcited by 13
- 2024Evaluating Numerical Reasoning in Text-to-Image Modelscited by 12
- 2024Confidence-aware Reward Optimization for Fine-tuning Text-to-Image Modelscited by 9
- 2026Understanding Reward Hacking in Text-to-Image Reinforcement Learningcited by 9
- 2026SpatialReward: Verifiable Spatial Reward Modeling for Fine-Grained Spatial Consistency in Text-to-Image Generationcited by 4
- 2026AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in UMMs via Decompositional Verifiable Rewardcited by 3
- 2026GeoSVG-RL: Geometry-Aware Reinforcement Learning for Layout-Constrained Text-to-SVG Diagram Generationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026ExploreNet: Learning Where to Explore in Diffusion GRPOcited by 0
- 2026ExploreNet: Learning Where to Explore in Diffusion GRPO29 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
- 2026Video Models Can Reason with Verifiable Rewards
- 2026Trace: A Taxonomy-Guided Environment for Multidomain Visual Reasoning
- 2026Same Reward, Different Skills: When Multimodal RL Learns to Look
- 2025SPHINX: A Synthetic Environment for Visual Perception and Reasoning
- 2026Visual-ERM: Reward Modeling for Visual Equivalence
- 2026Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models
- 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs