mlsanityData to 14 Oct 2026

SITE: towards Spatial Intelligence Thorough Evaluation

arXiv 2505.05456 ↗First posted 8 May 2025cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships, underpinning disciplines from neuroscience to robotics. We introduce SITE, a benchmark dataset towards SI Thorough Evaluation in a standardized format of multi-choice visual question-answering, designed to assess large vision-language models' spatial intelligence across diverse visual modalities (single-image, multi-image, and video) and SI factors (figural to environmental scales, spatial visualization and orientation, intrinsic and extrinsic, static and dynamic). Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science, which prompt us to design two novel types of tasks about view-taking and dynamic scenes. Extensive experiments reveal that leading models fall behind human experts especially in spatial orientation, a fundamental SI factor. Moreover, we demonstrate a positive correlation between a model's spatial reasoning proficiency and its performance on an embodied AI task.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 62 of its 91 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

62 of 91 references.

References · 62

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
38 more
  1. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  2. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  3. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  4. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  5. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  6. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  7. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  8. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  9. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  10. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  11. 2016Modeling Context in Referring Expressionscited by 1,063
  12. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  13. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 940
  14. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  15. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  16. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  17. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  18. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  19. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  20. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  21. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  22. 2018TVQA: Localized, Compositional Video Question Answeringcited by 467
  23. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 452
  24. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  25. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  26. 2022Visual Language Maps for Robot Navigationcited by 365
  27. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  28. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  29. 2022Visual Spatial Reasoningcited by 293
  30. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  31. 2021IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoningcited by 267
  32. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  33. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  34. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
  35. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  36. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  37. 2024MuirBench: A Comprehensive Benchmark for Robust Multi-image Understandingcited by 188
  38. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 169

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 21.

29 papers.

Sort:

By area:

  1. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 134
  2. 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 82
  3. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  4. 2025Holistic Evaluation of Multimodal LLMs on Spatial Intelligencecited by 29
  5. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligencecited by 24
  6. 2025Mull-Tokens: Modality-Agnostic Latent Thinkingcited by 19
  7. 2026Thinking with Geometry: Active Geometry Integration for Spatial Reasoningcited by 18
  8. 2026HY-Embodied-0.5: Embodied Foundation Models for Real-World Agentscited by 18
  9. 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 12
  10. 2026From Pixels to Words -- Towards Native One-Vision Models at Scalecited by 8
  11. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 8
  12. 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspectivecited by 7
17 more
  1. 2026Boosting MLLM Spatial Reasoning with Geometrically Referenced 3D Scene Representationscited by 5
  2. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 4
  3. 2026MultiView-Bench: A Diagnostic Benchmark for World-Centric Multi-View Integration in VLMscited by 4
  4. 2026Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligencecited by 3
  5. 2026Stepping VLMs onto the Court: Benchmarking Spatial Intelligence in Sportscited by 2
  6. 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMscited by 1
  7. 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
  8. 2026ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordinationcited by 1
  9. 2026PinpointQA: A Benchmark for Small Object-Centric Spatial Understanding in Indoor Videoscited by 0
  10. 2026Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matchingcited by 0
  11. 2026OrientSAM: Mitigating Camera-Centric Shortcut in Multimodal Spatial Reasoning via Orientation-Aware Spatial Alignmentcited by 0
  12. 2026ChainSpace: A Chained-Reasoning Paradigm for Spatial Intelligencecited by 0
  13. 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligencecited by 0
  14. 2026Metric-Bench: Exploring In-context Spatial Metric Reasoning in VLMs for Indoor Scenescited by 0
  15. 2026SpatialASM: Internalizing Active Mentalization for Spatial Reasoningcited by 0
  16. 2026GeoThinker-F: Endogenous Predictive World Representations for Fast Spatial Intelligencecited by 0
  17. 2026DynThinker: Unifying Static 3D and Dynamic 4D Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  2. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  3. 2025Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models
  4. 2026SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models
  5. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  6. 2025Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics
  7. 2026Thinking in Structures: Evaluating Spatial Intelligence in Constraint-Governed Spaces
  8. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?