# R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation

arXiv: https://arxiv.org/abs/2505.23493 · HTML: https://mlsanity.com/paper/2505.23493/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 56 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 30 of 39)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation](https://mlsanity.com/paper/2406.06525/) (2024)
- [Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality](https://mlsanity.com/paper/2204.03162/) (2022)
- [ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment](https://mlsanity.com/paper/2403.05135/) (2024)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Generative Multimodal Models are In-Context Learners](https://mlsanity.com/paper/2312.13286/) (2023)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation](https://mlsanity.com/paper/2312.14867/) (2023)
- [T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT](https://mlsanity.com/paper/2505.00703/) (2025)
- [SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer](https://mlsanity.com/paper/2501.18427/) (2025)
- [Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers](https://mlsanity.com/paper/2405.05945/) (2024)
- [GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation](https://mlsanity.com/paper/2406.13743/) (2024)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing](https://mlsanity.com/paper/2504.02826/) (2025)
- [Lumina-Image 2.0: A Unified and Efficient Image Generative Framework](https://mlsanity.com/paper/2503.21758/) (2025)
- [LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation](https://mlsanity.com/paper/2305.11116/) (2023)
- [Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration](https://mlsanity.com/paper/2305.14324/) (2023)
- [MagicMix: Semantic Mixing with Diffusion Models](https://mlsanity.com/paper/2210.16056/) (2022)
- [ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty](https://mlsanity.com/paper/2408.14339/) (2024)
- [Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?](https://mlsanity.com/paper/2406.07546/) (2024)
- [VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modeling](https://mlsanity.com/paper/2408.01181/) (2024)
- [ImageGen-CoT: Enhancing Text-to-Image In-context Learning with Chain-of-Thought Reasoning](https://mlsanity.com/paper/2503.19312/) (2025)

## Built on by (most cited first, 30 of 32)

- [TIIF-Bench: How Does Your T2I Model Follow Your Instructions?](https://mlsanity.com/paper/2506.02161/) (2025)
- [GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluation](https://mlsanity.com/paper/2512.16853/) (2025)
- [T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation](https://mlsanity.com/paper/2508.17472/) (2025)
- [Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?](https://mlsanity.com/paper/2509.03516/) (2025)
- [Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation](https://mlsanity.com/paper/2605.28091/) (2026)
- [Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image](https://mlsanity.com/paper/2512.16899/) (2025)
- [ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation](https://mlsanity.com/paper/2511.11483/) (2025)
- [Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models](https://mlsanity.com/paper/2601.20354/) (2026)
- [BizGenEval: A Systematic Benchmark for Commercial Visual Content Generation](https://mlsanity.com/paper/2603.25732/) (2026)
- [Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward](https://mlsanity.com/paper/2511.20561/) (2025)
- [SuperFlow: Training Flow Matching Models with RL on the Fly](https://mlsanity.com/paper/2512.17951/) (2025)
- [Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility](https://mlsanity.com/paper/2601.17027/) (2026)
- [AtelierEval: Agentic Evaluation of Humans & LLMs as Text-to-Image Prompters](https://mlsanity.com/paper/2605.22645/) (2026)
- [Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learning](https://mlsanity.com/paper/2512.09924/) (2025)
- [Beyond Words and Pixels: A Benchmark for Implicit World Knowledge Reasoning in Generative Models](https://mlsanity.com/paper/2511.18271/) (2025)
- [FireScope: Wildfire Risk Raster Prediction with a Chain-of-Thought Oracle](https://mlsanity.com/paper/2511.17171/) (2025)
- [AEGIS: Exploring the Limit of World Knowledge Capabilities for Unified Mulitmodal Models](https://mlsanity.com/paper/2601.00561/) (2026)
- [Agentic Flow Steering and Parallel Rollout Search for Spatially Grounded Text-to-Image Generation](https://mlsanity.com/paper/2603.18627/) (2026)
- [Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models](https://mlsanity.com/paper/2604.21523/) (2026)
- [SciIR: A Large-scale Training Dataset and Benchmark for Scientific Image Reasoning Generation](https://mlsanity.com/paper/2606.30124/) (2026)
- [Benchmarking Frontier Text-to-Image Models on Image-Description Prompts](https://mlsanity.com/paper/2608.14976/) (2026)
- [GenScale: A Benchmark for Relative Object Scale in Image Generation and Editing](https://mlsanity.com/paper/2609.00525/) (2026)
- [Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation](https://mlsanity.com/paper/2609.02864/) (2026)
- [Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?](https://mlsanity.com/paper/2603.00166/) (2026)
- [ProHiFlo: Hierarchical Flow Matching with Functional Guidance for De Novo Protein Generation](https://mlsanity.com/paper/2606.11243/) (2026)
- [FinInvest-GTCN: Explainable Graph-Temporal-Causal Modeling for Risk-Aware Investment Decision Optimization](https://mlsanity.com/paper/2606.28933/) (2026)
- [A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions](https://mlsanity.com/paper/2606.28943/) (2026)
- [ProWAFT: A ROMA-LPD Instance for Workload-Aware and Dynamic Fault Tolerance in FPGA-Based CNN Accelerators](https://mlsanity.com/paper/2607.01602/) (2026)
- [Graph-Based Modeling of Financial Volatility Dynamics](https://mlsanity.com/paper/2608.26127/) (2026)
- [Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models](https://mlsanity.com/paper/2609.02502/) (2026)

## Abstract (arXiv)

Reasoning is a fundamental capability often required in real-world text-to-image (T2I) generation, e.g., generating ``a bitten apple that has been left in the air for more than a week`` necessitates understanding temporal decay and commonsense concepts. While recent T2I models have made impressive progress in producing photorealistic images, their reasoning capability remains underdeveloped and insufficiently evaluated. To bridge this gap, we introduce R2I-Bench, a comprehensive benchmark specifically designed to rigorously assess reasoning-driven T2I generation. R2I-Bench comprises meticulously curated data instances, spanning core reasoning categories, including commonsense, mathematical, logical, compositional, numerical, causal, and concept mixing. To facilitate fine-grained evaluation, we design R2IScore, a QA-style metric based on instance-specific, reasoning-oriented evaluation questions that assess three critical dimensions: text-image alignment, reasoning accuracy, and image quality. Extensive experiments with 16 representative T2I models, including a strong pipeline-based framework that decouples reasoning and generation using the state-of-the-art language and image generation models, demonstrate consistently limited reasoning performance, highlighting the need for more robust, reasoning-aware architectures in the next generation of T2I systems. Project Page: https://r2i-bench.github.io