mlsanityData to 14 Oct 2026

Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics

arXiv 2502.11859 ↗First posted 17 Feb 2025cs.CV cs.CLcited by 23

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The Theory of Multiple Intelligences underscores the hierarchical nature of cognitive capabilities. To advance Spatial Artificial Intelligence, we pioneer a psychometric framework defining five Basic Spatial Abilities (BSAs) in Visual Language Models (VLMs): Spatial Perception, Spatial Relation, Spatial Orientation, Mental Rotation, and Spatial Visualization. Benchmarking 13 mainstream VLMs through nine validated psychometric experiments reveals significant gaps versus humans (average score 24.95 vs. 68.38), with three key findings: 1) VLMs mirror human hierarchies (strongest in 2D orientation, weakest in 3D rotation) with independent BSAs (Pearson's r<0.4); 2) Smaller models such as Qwen2-VL-7B surpass larger counterparts, with Qwen leading (30.82) and InternVL2 lagging (19.6); 3) Interventions like chain-of-thought (0.100 accuracy gain) and 5-shot training (0.259 improvement) show limits from architectural constraints. Identified barriers include weak geometry encoding and missing dynamic simulation. By linking psychometric BSAs to VLM capabilities, we provide a diagnostic toolkit for spatial intelligence evaluation, methodological foundations for embodied AI development, and a cognitive science-informed roadmap for achieving human-like spatial intelligence.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 6 of its 52 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

6 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning21 Oct

Other references · 5

  1. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  2. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  3. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  4. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  5. 2023Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 7.

23 papers.

Sort:

By area:

  1. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 65
  2. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
  3. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Modelscited by 30
  4. 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Sciencecited by 12
  5. 2025Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependencycited by 11
  6. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 11
  7. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysiscited by 9
  8. 2025Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Inputcited by 9
  9. 2025LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Taskscited by 6
  10. 2025Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMscited by 5
  11. 2025Visual serial processing deficits explain divergences in human and VLM reasoningcited by 4
  12. 2025GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Taskscited by 4
11 more
  1. 2025Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiationcited by 3
  2. 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modelingcited by 3
  3. 2025CubeBench: Diagnosing Interactive, Long-Horizon Spatial Reasoning Under Partial Observationscited by 3
  4. 2025DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometrycited by 1
  5. 2026PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoningcited by 1
  6. 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
  7. 2026From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISPcited by 1
  8. 2026MentalBlackboard: Evaluating Spatial Visualization via Mathematical Transformationscited by 0
  9. 2026Visual Prompt Discovery via Semantic Explorationcited by 0
  10. 2026Agentic AI with Structured CoT for Enhancing AI's Spatial Intelligence: Visualization and Reasoning of Rotationcited by 0
  11. 2026Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  2. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  3. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
  4. 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests
  5. 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
  6. 2026ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models
  7. 2025SITE: towards Spatial Intelligence Thorough Evaluation
  8. 2025MindCube: Spatial Mental Modeling from Limited Views