Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?
Not on the map. Why?
Abstract (arXiv)
Text-to-image (T2I) generation aims to synthesize images from textual prompts, which jointly specify what must be shown and imply what can be inferred, which thus correspond to two core capabilities: textbftextitcomposition and textbftextitreasoning. Despite recent advances of T2I models in both composition and reasoning, existing benchmarks remain limited in evaluation. They not only fail to provide comprehensive coverage across and within both capabilities, but also largely restrict evaluation to low scene density and simple one-to-one reasoning. To address these limitations, we propose textbftextscT2I-CoReBench, a comprehensive and complex benchmark that evaluates both composition and reasoning capabilities of T2I models. To ensure comprehensiveness, we structure composition around scene graph elements (textitinstance, textitattribute, and textitrelation) and reasoning around the philosophical framework of inference (textitdeductive, textitinductive, and textitabductive), formulating a 12-dimensional evaluation taxonomy. To increase complexity, driven by the inherent real-world complexities, we curate each prompt with higher compositional density for composition and greater reasoning intensity for reasoning. To facilitate fine-grained and reliable evaluation, we also pair each evaluation prompt with a checklist that specifies individual textityes/no questions to assess each intended element independently. In statistics, our benchmark comprises 1,080 challenging prompts and around 13,500 checklist questions. Experiments across 38 current T2I models reveal that their composition capability still remains limited in high compositional scenarios, while the reasoning capability lags even further behind as a critical bottleneck, with all models struggling to infer implicit elements from prompts.
Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 67 of its 99 references that are in the map. First posted on arXiv 3 Sep 2025.
Builds on
67 of 99 references.
References · 67
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Visual Instruction Tuningcited by 7,869
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2021Zero-Shot Text-to-Image Generationcited by 4,288
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
38 more
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
- 2025Qwen-Image Technical Reportcited by 1,044
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2024Autoregressive Image Generation without Vector Quantizationcited by 651
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
- 2024A Sanity Check for AI-generated Image Detectioncited by 189
- 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
- 2025Seedream 3.0 Technical Reportcited by 154
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 141
- 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
- 2024Evaluation of OpenAI o1: Opportunities and Challenges of AGIcited by 127
- 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 127
- 2021A Comprehensive Survey of Scene Graphs: Generation and Applicationcited by 125
- 2023What You See is What You Read? Improving Text-Image Alignment Evaluationcited by 108
- 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 92
- 2024Improving Synthetic Image Detection Towards Generalization: An Image Transformation Perspectivecited by 89
- 2025OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generationcited by 87
- 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 71
- 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?cited by 69
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2025: 7; 2026: 27.
34 papers.
Sort:
By area:
- 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
- 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Modelscited by 16
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
- 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editingcited by 14
- 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligencecited by 13
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 12
- 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 12
- 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2026MetaPoint: Unlocking Precise Spatial Control in Agentic Visual Generationcited by 4
- 2026Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Modelcited by 3
22 more
- 2026SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosiscited by 3
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insightscited by 2
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generationcited by 2
- 2026OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibrationcited by 2
- 2026ForensicConcept: Transferable Forensic Concepts for AIGI Detectioncited by 2
- 2025Chain-of-Image Generation: Toward Monitorable and Controllable Image Generationcited by 1
- 2026Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?cited by 1
- 2026MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generationcited by 1
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
- 2026DynEval: Holistic Evaluations of T2I Generative Models in the Wildcited by 1
- 2026GlobalForge: Towards Robust AI-Generated Image Detectioncited by 1
- 2026PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoderscited by 1
- 2026Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I modelscited by 0
- 2026DECODE: Tackling Representation and Decision Degradation in Continual AI-Generated Image Detectioncited by 0
- 2026TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generationcited by 0
- 2026LoRC: Detecting AI-Generated Images via Low-Rank Collapse in Semantic Residualscited by 0
- 2026Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routingcited by 0
- 2026Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generationcited by 0
- 2026SolveEdit: Benchmarking Visual Problem Solving in Generative Modelscited by 0
- 2026PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latentscited by 0
- 2026Think Before You Score: Thinking Reward Model for Visual Generationcited by 0
- 2026Looped Diffusion Transformercited by 0
- 2026Looped Diffusion Transformer30 Sep 2026
- 2026Think Before You Score: Thinking Reward Model for Visual Generation29 Sep 2026
- 2026PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents28 Sep 2026
- 2026SolveEdit: Benchmarking Visual Problem Solving in Generative Models28 Sep 2026
- 2026Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generation26 Sep 2026
- 2026Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing25 Aug 2026
- 2026LoRC: Detecting AI-Generated Images via Low-Rank Collapse in Semantic Residuals21 Aug 2026
- 2026TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation17 Aug 2026
- 2026PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoders11 Aug 2026
- 2026DECODE: Tackling Representation and Decision Degradation in Continual AI-Generated Image Detection30 Jul 2026
- 2026GlobalForge: Towards Robust AI-Generated Image Detection16 Jul 2026
- 2026DynEval: Holistic Evaluations of T2I Generative Models in the Wild13 Jul 2026
22 more
- 2026ForensicConcept: Transferable Forensic Concepts for AIGI Detection5 Jun 2026
- 2026Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models4 Jun 2026
- 2026MetaPoint: Unlocking Precise Spatial Control in Agentic Visual Generation3 Jun 2026
- 2026OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration27 May 2026
- 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation5 May 2026
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models13 Apr 2026
- 2026MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation30 Mar 2026
- 2026Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?27 Feb 2026
- 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editing12 Feb 2026
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation12 Feb 2026
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Models2 Feb 2026
- 2026SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosis30 Jan 2026
- 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models28 Jan 2026
- 2026Thinking with Frames: Generative Video Distortion Evaluation via Frame Reward Model7 Jan 2026
- 2025Chain-of-Image Generation: Toward Monitorable and Controllable Image Generation9 Dec 2025
- 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence2 Dec 2025
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights1 Dec 2025
- 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing29 Nov 2025
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmark3 Nov 2025
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoning13 Oct 2025
- 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark29 Sep 2025
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
- 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligencecited by 13
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generationcited by 2
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models13 Apr 2026
- 2026Spatial Chain-of-Thought: Bridging Understanding and Generation Models for Spatial Reasoning Generation12 Feb 2026
- 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence2 Dec 2025
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoning13 Oct 2025
- 2026PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoderscited by 1
- 2026PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latentscited by 0
- 2026Looped Diffusion Transformercited by 0
- 2026Looped Diffusion Transformer30 Sep 2026
- 2026PreviewDiff: Multimodal Critic-Guided Search over Diffusion Latents28 Sep 2026
- 2026PEAK: Precise and Persistent Concept Erasure via k-Sparse Autoencoders11 Aug 2026
- 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
- 2025UniREditBench: A Unified Reasoning-based Image Editing Benchmarkcited by 25
- 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Modelscited by 16
- 2026DeepGen 1.0: A Lightweight Unified Multimodal Model for Advancing Image Generation and Editingcited by 14
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 12
- 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 12
- 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11
- 2026SANEval: Open-Vocabulary Compositional Benchmarks with Failure-mode Diagnosiscited by 3
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insightscited by 2
- 2026ForensicConcept: Transferable Forensic Concepts for AIGI Detectioncited by 2
- 2025Chain-of-Image Generation: Toward Monitorable and Controllable Image Generationcited by 1
- 2026MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generationcited by 1
12 most cited of 21 All 21 (JSON)
- 2026SolveEdit: Benchmarking Visual Problem Solving in Generative Models28 Sep 2026
- 2026Presence Is Not Faithfulness: Figurative Vehicle Intrusion in Text-to-Image Generation26 Sep 2026
- 2026Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing25 Aug 2026
- 2026LoRC: Detecting AI-Generated Images via Low-Rank Collapse in Semantic Residuals21 Aug 2026
- 2026TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation17 Aug 2026
- 2026DECODE: Tackling Representation and Decision Degradation in Continual AI-Generated Image Detection30 Jul 2026
- 2026GlobalForge: Towards Robust AI-Generated Image Detection16 Jul 2026
- 2026DynEval: Holistic Evaluations of T2I Generative Models in the Wild13 Jul 2026
- 2026ForensicConcept: Transferable Forensic Concepts for AIGI Detection5 Jun 2026
- 2026Faithful, Enriched, and Precise: Benchmarking Natural-Science Illustration Generation by T2I models4 Jun 2026
- 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generation5 May 2026
- 2026MathGen: Revealing the Illusion of Mathematical Competence through Text-to-Image Generation30 Mar 2026
12 newest of 21 All 21 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026Iterative Refinement Improves Compositional Image Generation
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation
- 2025UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation
- 2025WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation
- 2024GraPE: A Generate-Plan-Edit Framework for Compositional T2I Synthesis
- 2026EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation
- 2026Evaluating Reasoning Fidelity in Visual Text Generation
- 2025T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation