# Defining and Evaluating Visual Language Models' Basic Spatial Abilities: A Perspective from Psychometrics

arXiv: https://arxiv.org/abs/2502.11859 · HTML: https://mlsanity.com/paper/2502.11859/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 6 of its 52 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 6 of 6)

- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoning](https://mlsanity.com/paper/2403.11401/) (2024)
- [Evaluating Cognitive Maps and Planning in Large Language Models with CogEval](https://mlsanity.com/paper/2309.15129/) (2023)
- [Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning](https://mlsanity.com/paper/2410.16162/) (2024)
- [Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs](https://mlsanity.com/paper/2304.11164/) (2023)

## Built on by (most cited first, 23 of 23)

- [Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views](https://mlsanity.com/paper/2510.18632/) (2025)
- [UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces](https://mlsanity.com/paper/2503.06157/) (2025)
- [Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2503.19707/) (2025)
- [A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science](https://mlsanity.com/paper/2504.09848/) (2025)
- [Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency](https://mlsanity.com/paper/2504.18589/) (2025)
- [SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery](https://mlsanity.com/paper/2512.07733/) (2025)
- [11Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis](https://mlsanity.com/paper/2508.20068/) (2025)
- [Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input](https://mlsanity.com/paper/2510.16926/) (2025)
- [LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks](https://mlsanity.com/paper/2507.20174/) (2025)
- [Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMs](https://mlsanity.com/paper/2507.11932/) (2025)
- [Visual serial processing deficits explain divergences in human and VLM reasoning](https://mlsanity.com/paper/2509.25142/) (2025)
- [GamiBench: Evaluating Spatial Reasoning and 2D-to-3D Planning Capabilities of MLLMs with Origami Folding Tasks](https://mlsanity.com/paper/2512.22207/) (2025)
- [Spatial CAPTCHA: Generatively Benchmarking Spatial Reasoning for Human-Machine Differentiation](https://mlsanity.com/paper/2510.03863/) (2025)
- [Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling](https://mlsanity.com/paper/2512.01821/) (2025)
- [CubeBench: Diagnosing Interactive, Long-Horizon Spatial Reasoning Under Partial Observations](https://mlsanity.com/paper/2512.23328/) (2025)
- [DynaSolidGeo: A Dynamic Benchmark for Genuine Spatial Mathematical Reasoning of VLMs in Solid Geometry](https://mlsanity.com/paper/2510.22340/) (2025)
- [PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning](https://mlsanity.com/paper/2605.19382/) (2026)
- [SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes](https://mlsanity.com/paper/2605.31148/) (2026)
- [From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP](https://mlsanity.com/paper/2606.26535/) (2026)
- [MentalBlackboard: Evaluating Spatial Visualization via Mathematical Transformations](https://mlsanity.com/paper/2602.19357/) (2026)
- [Visual Prompt Discovery via Semantic Exploration](https://mlsanity.com/paper/2603.16250/) (2026)
- [Agentic AI with Structured CoT for Enhancing AI's Spatial Intelligence: Visualization and Reasoning of Rotation](https://mlsanity.com/paper/2610.04188/) (2026)
- [Fine-Tuning VLM for Enhancing AI's Spatial Intelligence: Understanding 3D and 2D Rotations](https://mlsanity.com/paper/2610.04206/) (2026)

## Abstract (arXiv)

The Theory of Multiple Intelligences underscores the hierarchical nature of cognitive capabilities. To advance Spatial Artificial Intelligence, we pioneer a psychometric framework defining five Basic Spatial Abilities (BSAs) in Visual Language Models (VLMs): Spatial Perception, Spatial Relation, Spatial Orientation, Mental Rotation, and Spatial Visualization. Benchmarking 13 mainstream VLMs through nine validated psychometric experiments reveals significant gaps versus humans (average score 24.95 vs. 68.38), with three key findings: 1) VLMs mirror human hierarchies (strongest in 2D orientation, weakest in 3D rotation) with independent BSAs (Pearson's r<0.4); 2) Smaller models such as Qwen2-VL-7B surpass larger counterparts, with Qwen leading (30.82) and InternVL2 lagging (19.6); 3) Interventions like chain-of-thought (0.100 accuracy gain) and 5-shot training (0.259 improvement) show limits from architectural constraints. Identified barriers include weak geometry encoding and missing dynamic simulation. By linking psychometric BSAs to VLM capabilities, we provide a diagnostic toolkit for spatial intelligence evaluation, methodological foundations for embodied AI development, and a cognitive science-informed roadmap for achieving human-like spatial intelligence.