mlsanityData to 14 Oct 2026

SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition

arXiv 2511.21471 ↗First posted 26 Nov 2025cs.AIcited by 35

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Spatial cognition is fundamental to real-world multimodal intelligence, allowing models to effectively interact with the physical environment. While multimodal large language models (MLLMs) have made significant strides, existing benchmarks often oversimplify spatial cognition, reducing it to a single-dimensional metric, which fails to capture the hierarchical structure and interdependence of spatial abilities. To address this gap, we propose a hierarchical spatial cognition framework that decomposes spatial intelligence into five progressively complex levels from basic observation to high-level planning. Building upon this taxonomy, we construct SpatialBench, a large-scale, fine-grained benchmark covering 15 tasks aligned with these cognitive levels. To provide a unified evaluation across heterogeneous tasks, we further introduce a high-level capability-oriented metric that reliably assesses a model's overall spatial reasoning ability. Extensive experiments over massive MLLMs reveal distinct performance stratification across cognitive levels: models exhibit strong perceptual grounding yet remain limited in symbolic reasoning, causal inference, and planning. Additional human tests demonstrate that humans perform selective, goal-directed abstraction, while MLLMs tend to over-attend to surface details without coherent spatial intent. Our work establishes the first systematic framework for measuring hierarchical spatial cognition in MLLMs, laying the foundation for future spatially intelligent systems.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 67 of its 81 references that are in the map. First posted on arXiv 26 Nov 2025.

Builds on

67 of 81 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models9 Oct
  2. 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Models4 Aug
  3. 2025MindCube: Spatial Mental Modeling from Limited Views26 Jun
  4. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence29 May
  5. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
  6. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?31 Mar
  7. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D29 Mar
  8. 2025ST-Think: How Multimodal Large Language Models Reason About 4D Worlds from Ego-Centric Videos16 Mar
  9. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models2 Jan
  10. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
  11. 2024LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness26 Sep
  12. 2024VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs2 Jul
6 more
  1. 2024SpatialBot: Precise Spatial Understanding with Vision Language Models19 Jun
  2. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasoners4 Jun
  3. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
  4. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  5. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planning30 Nov
  6. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 49

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  9. 2024GPT-4o System Cardcited by 4,588
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
37 more
  1. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  2. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  7. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  8. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  10. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  11. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  12. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  13. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  14. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  15. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  16. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  17. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 528
  18. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  19. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  20. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  21. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 299
  22. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  23. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  24. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 229
  25. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
  26. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  27. 2025MiniMax-01: Scaling Foundation Models with Lightning Attentioncited by 166
  28. 2023Osprey: Pixel Understanding with Visual Instruction Tuningcited by 159
  29. 2024Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolutioncited by 159
  30. 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
  31. 2025Visual Spatial Tuningcited by 89
  32. 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentationcited by 70
  33. 2023Evaluating Spatial Understanding of Large Language Modelscited by 66
  34. 2024Does Spatial Cognition Emerge in Frontier Models?cited by 66
  35. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selectioncited by 65
  36. 2023Diversity-Aware Meta Visual Promptingcited by 58
  37. 2017Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017cited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 34.

35 papers.

Sort:

By area:

  1. 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoningcited by 17
  2. 2026SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Taskscited by 7
  3. 2026Spatial-Omni: Spatial Audio Understanding Integration in Multimodal LLMs via FOA Encodingcited by 7
  4. 2026CapNav: Benchmarking Vision Language Models on Capability-conditioned Indoor Navigationcited by 6
  5. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
  6. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agentscited by 5
  7. 2026SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environmentscited by 5
  8. 2026TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoningcited by 4
  9. 2026GPSBench: Do Large Language Models Understand GPS Coordinates?cited by 4
  10. 2026See, Remember, Explore: A Benchmark and Baselines for Streaming Spatial Reasoningcited by 4
  11. 2026CRAFT: Grounded Multi-Agent Coordination Under Partial Informationcited by 3
  12. 2026SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learningcited by 3
23 more
  1. 2026CVT-Bench: Probing Spatial-State Integrity through Counterfactual Viewpoint Transformationscited by 2
  2. 20263D Primitives are a Spatial Language for VLMscited by 2
  3. 2026Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?cited by 2
  4. 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
  5. 2026Predicting Camera Pose from Perspective Descriptions for Spatial Reasoningcited by 1
  6. 2026From Pixels to BFS: High Maze Accuracy Does Not Imply Visual Planningcited by 1
  7. 2026Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Modelscited by 1
  8. 2026Visual Credit Audit for Multimodal Spatial Reasoningcited by 1
  9. 2026CoMa: Contextual Massing Generation with Vision-Language Modelscited by 0
  10. 2026SPATIALALIGN: Aligning Dynamic Spatial Relationships in Video Generationcited by 0
  11. 2026SGR3 Model: Scene Graph Retrieval-Reasoning Model in 3Dcited by 0
  12. 2026Hidden Meanings in Plain Sight: RebusBench for Evaluating Cognitive Visual Reasoningcited by 0
  13. 2026TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tablescited by 0
  14. 2026ARGOS: Who, Where, and When in Agentic Multi-Camera Person Searchcited by 0
  15. 2026SocialGrid: A Benchmark for Planning and Social Reasoning in Embodied Multi-Agent Systemscited by 0
  16. 2026Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoTcited by 0
  17. 2026Brick-Composer: Using MLLMs for Assembly with Diverse Brickscited by 0
  18. 2026OmniCoT: A Benchmark for Global and Multi-Step Panoramic Reasoningcited by 0
  19. 2026IDEAL-Bench: Indoor Dataset and Evaluation suite for Analyzing 3D Layout reasoningcited by 0
  20. 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Testscited by 0
  21. 2026PolyComp: A Polycube-based Benchmark for Compositional 3D Spatial Reasoning in Multimodal Modelscited by 0
  22. 2026MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agentscited by 0
  23. 2026Does Local Video Understanding Transfer Across Encounters? The EgoGears Benchmarkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  2. 202511Plus-Bench: Demystifying Multimodal LLM Spatial Reasoning with Cognitive-Inspired Analysis
  3. 2025Spatial4D-Bench: A Versatile 4D Spatial Intelligence Benchmark
  4. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence
  5. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
  6. 2026From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
  7. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  8. 2026SpatialText: A Pure-Text Cognitive Benchmark for Spatial Understanding in Large Language Models