Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text
Not on the map. Why?
Abstract (arXiv)
Spatial intelligence is essential for agents to move from static semantic understanding toward interacting with the physical world. Many spatial tasks are grounded in continuous visual scenes, where locations, regions, and paths are more naturally expressed by pointing, marking, or drawing than by reporting precise coordinates or discrete textual symbols. Yet existing spatial reasoning benchmarks usually require coordinates, options, or text, creating an answer-interface mismatch for image-generation models. This makes it difficult to evaluate image-generation models under the same task semantics as text-output VLMs, despite their ability to externalize spatial judgments directly in pixel space. We propose ProVisE (Protocolized Visual Evaluation), a benchmark-agnostic framework that elicits protocol-constrained visual answers from image-generation models and parses them into structured predictions compatible with original metrics. ProVisE also includes an Agentic builder that constructs and validates task-specific protocols for new benchmarks. We further introduce SpatialGen-Bench, a curated diagnostic benchmark of 470 samples across 14 spatial subtasks, four capability levels, and diverse answer forms. We evaluate representative text-output VLMs and image-generation models in a unified setting and validate Agentic protocol construction on six external spatial benchmarks. Results show that image-generation models are competitive when spatial answers can be externalized directly in pixel space, while text-output VLMs retain a clear advantage in compositional spatial reasoning. These findings reveal complementary strengths of pixel-space expression and text-based reasoning and establish a metric-compatible testbed for studying spatial cognition in image-generation models.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 56 of its 61 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
56 of 61 references.
References · 56
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
38 more
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2022Visual Spatial Reasoningcited by 288
- 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2023Teaching CLIP to Count to Tencited by 194
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
- 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 159
- 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
- 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 146
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 137
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concretecited by 134
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
- 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 130
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
- 2025Emu3.5: Native Multimodal Models are World Learnerscited by 119
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 111
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 110
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Modelscited by 100
- 2022Benchmarking Spatial Relationships in Text-to-Image Generationcited by 96
- 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Modelscited by 94
- 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
- 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Modelscited by 54
- 2025SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoningcited by 45
- 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
- 2026Image Generators are Generalist Vision Learnerscited by 40
- 2026RoboBrain 2.5: Depth in Sight, Time in Mindcited by 38
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognitioncited by 35
- 2026RynnBrain: Open Embodied Foundation Modelscited by 33
- 2024SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluationcited by 31
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 31
- 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligencecited by 19
- 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Modelscited by 18
- 2025RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigationcited by 17
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026Image-Space Rule Discoverycited by 1
- 2026What Looks Like a Capability Limit in Vision-Language Models Is a Readout Limitcited by 0
- 2026What Looks Like a Capability Limit in Vision-Language Models Is a Readout Limit23 Sep 2026
- 2026Image-Space Rule Discovery1 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2025SITE: towards Spatial Intelligence Thorough Evaluation
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
- 2026SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
- 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
- 2026When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
- 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions