From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Current VLM evaluations often conflate language priors with genuine spatial reasoning. To address this, we introduce CRISP, a novel structural-diagnostic evaluation paradigm that assesses visual spatial intelligence through consistency, the alignment between implicit perception and explicit reasoning. Unlike traditional black-box QA, CRISP utilizes metric 3D Scene Graphs and an oracle intervention protocol to decouple latent reasoning capabilities from perceptual bottlenecks. This granular diagnosis uncovers a systematic perception-reasoning disconnect. Crucially, we reveal that while proprietary models possess robust latent reasoning engines, they suffer from inaccurate metric estimation and a critical failure to leverage their implicit structural representations. Conversely, open-source models remain fundamentally bottlenecked by their lack of multi-hop compositional reasoning. By shifting the focus from merely ``guessing correctly'' via language priors to genuinely ``perceiving, verifying, and reasoning,'' CRISP offers a rigorous roadmap for multimodal alignment beyond end-to-end post-training. The code and dataset are available at https://github.com/iiyamayuki/CRISP-Bench.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 60 references that are in the map. First posted on arXiv 25 Jun 2026.
Builds on
51 of 60 references.
Closest ancestors · 12
Same direction, newest first
- 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective23 Sep
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models3 Jun
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence29 May
- 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning28 Apr
- 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoning2 Apr
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark10 Dec
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models21 Jun
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
- 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models4 Apr
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
- 2022SQA3D: Situated Question Answering in 3D Scenes14 Oct
Other references · 39
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Qwen Technical Reportcited by 3,273
- 2021Learning to Prompt for Vision-Language Modelscited by 2,269
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
27 more
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023LIMA: Less Is More for Alignmentcited by 1,080
- 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2025OpenAI GPT-5 System Cardcited by 910
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
- 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
- 2025LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Trainingcited by 233
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
- 20233D-VisTA: Pre-trained Transformer for 3D Vision and Text Alignmentcited by 201
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 132
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
- 2025Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priorscited by 111
- 2024NaturalBench: Evaluating Vision-Language Models on Natural Adversarial Samplescited by 94
- 2024Does Spatial Cognition Emerge in Frontier Models?cited by 65
- 2024TIPS: Text-Image Pretraining with Spatial awarenesscited by 37
- 2023Collaborative Dynamic 3D Scene Graphs for Automated Drivingcited by 34
- 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 26
- 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometricscited by 21
- 2026Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?cited by 18
- 2025Robust Multimodal Large Language Models Against Modality Conflictcited by 17
- 2025Towards Physics-informed Spatial Intelligence with Human Priors: An Autonomous Driving Pilot Studycited by 2
- 2025Latent Domain Prompt Learning for Vision-Language Modelscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective
- 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
- 2026Consistent Yet Wrong: Evidence Insensitivity in Spatial Vision-Language Models
- 2026When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning
- 2026Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning
- 2026Disentangling 3D Modeling from Spatial Reasoning
- 2026SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
- 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards