mlsanityData to 7 Oct 2026

Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation

arXiv 2609.02864 ↗First posted 2 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advancements in unified generative models (UGMs) and world simulators have achieved unprecedented results in visual perception and synthesis. However, these models primarily rely on surface-level event alignment, leaving the capacity for high-level visual reasoning underexplored. True visual generative intelligence demands "Reasoning-to-Generation", an ability to infer latent rules from visual inputs and manifest solutions through precise, logically constrained visual outcomes. We introduce RIG-BENCH, a novel comprehensive benchmark that systematically evaluates Reasoning-driven Image Generation (RIG) across four cognitively demanding domains: Concept-based, Transformation-based, Pattern & Structure, and Scenario-based. Featuring 2000 curated samples, RIG-BENCH serves as a rigorous stress test for RIG. Our extensive evaluations of state-of-the-art UGMs and image/video generation models reveal a significant reasoning-generation gap, wherein models frequently produce locally plausible but globally illogical outputs. RIG-BENCH provides a vital diagnostic framework to guide the development of next-generation, logically grounded UGMs and world simulators.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 54 of its 84 references that are in the map. First posted on arXiv 2 Sep 2026.

Builds on

54 of 84 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  8. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  10. 2025Qwen-Image Technical Reportcited by 973
  11. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  12. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
38 more
  1. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  2. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  3. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  4. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  5. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  6. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
  7. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  8. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  9. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  10. 2019RAVEN: A Dataset for Relational and Analogical Visual rEasoNingcited by 225
  11. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  12. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 171
  13. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
  14. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 132
  15. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  16. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 81
  17. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  18. 2026BabyVision: Visual Reasoning Beyond Languagecited by 39
  19. 2025T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generationcited by 38
  20. 2024PhyBench: A Physical Commonsense Benchmark for Evaluating Text-to-Image Modelscited by 36
  21. 2025R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generationcited by 31
  22. 2024What is the Visual Cognition Gap between Humans and Multimodal LLMs?cited by 30
  23. 2026A Very Big Video Reasoning Suitecited by 29
  24. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 28
  25. 2025MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Modelscited by 24
  26. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 22
  27. 2025V-ReasonBench: Toward Unified Reasoning Benchmark Suite for Video Generation Modelscited by 22
  28. 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generationcited by 18
  29. 2025GIR-Bench: Versatile Benchmark for Generating Images with Reasoningcited by 15
  30. 2025MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoningcited by 14
  31. 2025Human Cognitive Benchmarks Reveal Foundational Visual Gaps in MLLMscited by 12
  32. 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 11
  33. 2026UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generationcited by 11
  34. 2026UEval: A Benchmark for Unified Multimodal Generationcited by 9
  35. 2026MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoningcited by 8
  36. 2025Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problemscited by 7
  37. 2026MentisOculi: Revealing the Limits of Reasoning with Mental Imagerycited by 6
  38. 2025Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMscited by 5

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation
  2. 2026VGI-Bench: Probing Visual Intelligence in Video Generation Models
  3. 2026GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models
  4. 2025TiViBench: Benchmarking Think-in-Video Reasoning for Video Generative Models
  5. 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence
  6. 2026ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis
  7. 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
  8. 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark