Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation
Not on the map. Why?
Abstract (arXiv)
Recent advancements in unified generative models (UGMs) and world simulators have achieved unprecedented results in visual perception and synthesis. However, these models primarily rely on surface-level event alignment, leaving the capacity for high-level visual reasoning underexplored. True visual generative intelligence demands "Reasoning-to-Generation", an ability to infer latent rules from visual inputs and manifest solutions through precise, logically constrained visual outcomes. We introduce RIG-BENCH, a novel comprehensive benchmark that systematically evaluates Reasoning-driven Image Generation (RIG) across four cognitively demanding domains: Concept-based, Transformation-based, Pattern & Structure, and Scenario-based. Featuring 2000 curated samples, RIG-BENCH serves as a rigorous stress test for RIG. Our extensive evaluations of state-of-the-art UGMs and image/video generation models reveal a significant reasoning-generation gap, wherein models frequently produce locally plausible but globally illogical outputs. RIG-BENCH provides a vital diagnostic framework to guide the development of next-generation, logically grounded UGMs and world simulators.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 54 of its 84 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
54 of 84 references.
References · 54
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
- 2025Qwen-Image Technical Reportcited by 973
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
38 more
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
- 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 171
- 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 132
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
- 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
- 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
- 2026BabyVision: Visual Reasoning Beyond Languagecited by 39
- 2025T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generationcited by 38
- 2024PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Modelscited by 36
- 2025R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generationcited by 31
- 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
- 2026A Very Big Video Reasoning Suitecited by 29
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 28
- 2025MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Modelscited by 24
- 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 22
- 2025V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Modelscited by 22
- 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generationcited by 18
- 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
- 2025MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoningcited by 14
- 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMscited by 12
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 11
- 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationcited by 11
- 2026UEval: A Benchmark for Unified Multimodal Generationcited by 9
- 2026MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoningcited by 8
- 2025Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problemscited by 7
- 2026MentisOculi: Revealing the Limits of Reasoning with Mental Imagerycited by 6
- 2025Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMscited by 5
50 most cited of 54 All 54 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation
- 2026VGI-Bench: Probing Visual Intelligence in Video Generation Models
- 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models
- 2025TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models
- 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence
- 2026ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
- 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark