mlsanityData to 7 Oct 2026

An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models

arXiv 2411.06048 ↗First posted 9 Nov 2024cs.CV cs.AIcited by 49

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Large Multimodal Models (LMMs) have achieved strong performance across a range of vision and language tasks. However, their spatial reasoning capabilities are under-investigated. In this paper, we construct a novel VQA dataset, Spatial-MM, to comprehensively study LMMs' spatial understanding and reasoning capabilities. Our analyses on object-relationship and multi-hop reasoning reveal several important findings. Firstly, bounding boxes and scene graphs, even synthetic ones, can significantly enhance LMMs' spatial reasoning. Secondly, LMMs struggle more with questions posed from the human perspective than the camera perspective about the image. Thirdly, chain of thought (CoT) prompting does not improve model performance on complex multi-hop questions involving spatial relations. % Moreover, spatial reasoning steps are much less accurate than non-spatial ones across MLLMs. Lastly, our perturbation analysis on GQA-spatial reveals that LMMs are much stronger at basic object detection than complex spatial reasoning. We believe our benchmark dataset and in-depth analyses can spark further research on LMMs spatial reasoning. Spatial-MM benchmark is available at: https://github.com/FatemehShiri/Spatial-MM

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 32 references that are in the map. First posted on arXiv 9 Nov 2024.

Builds on

17 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  2. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct

Other references · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  11. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  12. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
3 more
  1. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 56
  2. 2024Automatic Programming: Large Language Models and Beyondcited by 35
  3. 2024Direct Evaluation of Chain-of-Thought in Multi-hop Reasoning with Knowledge Graphscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 33; 2026: 14.

49 papers.

Sort:

By area:

  1. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
  2. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  3. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 110
  4. 2025Baichuan-Omni-1.5 Technical Reportcited by 107
  5. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
  6. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  7. 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
  8. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
  9. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  10. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Spacecited by 37
  11. 2025SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Modelscited by 24
  12. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 20
37 more
  1. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenescited by 19
  2. 2026Current Agents Fail to Leverage World Model as Tool for Foresightcited by 15
  3. 2025Visual Structures Helps Visual Reasoning: Addressing the Binding Problem in VLMscited by 14
  4. 2026On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMscited by 11
  5. 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 10
  6. 2025Can World Models Benefit VLMs for World Dynamics?cited by 9
  7. 2025Flow2Code: Evaluating Large Language Models for Flowchart-based Code Generation Capabilitycited by 7
  8. 2025Seeing is Not Reasoning: MVPBench for Graph-based Evaluation of Multi-path Visual Physical CoTcited by 6
  9. 2025ViPlan: A Benchmark for Visual Planning with Symbolic Predicates and Vision-Language Modelscited by 5
  10. 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 5
  11. 2025Perceiving Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Modelscited by 4
  12. 2025Caption This, Reason That: VLMs Caught in the Middlecited by 4
  13. 2025LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Taskscited by 4
  14. 2025Scene Graph-Guided Proactive Replanning for Failure-Resilient Embodied Agentcited by 4
  15. 2025RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotationcited by 4
  16. 2025PENDULUM: A Benchmark for Assessing Sycophancy in Multimodal Large Language Modelscited by 4
  17. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 3
  18. 2026Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMscited by 3
  19. 2026Aligning What Vision-Language Models See and Perceive with Adaptive Information Flowcited by 2
  20. 2026DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Drivingcited by 2
  21. 2026SATURN: Symbolic Spatial Reasoning for Multi-Perspective Groundingcited by 2
  22. 2024Automatically Detecting Online Deceptive Patternscited by 1
  23. 2025CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Modelscited by 1
  24. 2025BYO-Eval: Build Your Own Dataset for Fine-Grained Visual Assessment of Multimodal Language Modelscited by 1
  25. 2025Understanding Space Is Rocket Science -- Only Top Reasoning Models Can Solve Spatial Understanding Taskscited by 1
  26. 2025MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlookcited by 1
  27. 2025Dynamic Orchestration of Multi-Agent System for Real-World Multi-Image Agricultural VQAcited by 1
  28. 2025GOPLA: Generalizable Object Placement Learning via Synthetic Augmentation of Human Arrangementcited by 1
  29. 2025M3-Verse: A "Spot the Difference" Challenge for Large Multimodal Modelscited by 1
  30. 2026HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Modelscited by 1
  31. 2026Generation as Auxiliary Supervision: Enhancing Visual Understanding at Zero Inference Overhead via Decoupled Embedding Predictioncited by 1
  32. 2025Spatial Knowledge Graph-Guided Multimodal Synthesiscited by 0
  33. 2025OceanGym: A Benchmark Environment for Underwater Embodied Agentscited by 0
  34. 2026State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Modelscited by 0
  35. 2026Why MLLMs Struggle to Determine Object Orientationscited by 0
  36. 2026Agent-Aided Design for Dynamic CAD Modelscited by 0
  37. 2026Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Codingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models
  2. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
  3. 2026Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
  4. 2024GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMs
  5. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  6. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  7. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  8. 2025Can Multimodal Large Language Models Understand Spatial Relations?