mlsanityData to 14 Oct 2026

Can Multimodal Large Language Models Understand Spatial Relations?

arXiv 2505.19015 ↗First posted 25 May 2025cs.CV cs.MMcited by 20

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Spatial relation reasoning is a crucial task for multimodal large language models (MLLMs) to understand the objective world. However, current benchmarks have issues like relying on bounding boxes, ignoring perspective substitutions, or allowing questions to be answered using only the model's prior knowledge without image understanding. To address these issues, we introduce SpatialMQA, a human-annotated spatial relation reasoning benchmark based on COCO2017, which enables MLLMs to focus more on understanding images in the objective world. To ensure data quality, we design a well-tailored annotation procedure, resulting in SpatialMQA consisting of 5,392 samples. Based on this benchmark, a series of closed- and open-source MLLMs are implemented and the results indicate that the current state-of-the-art MLLM achieves only 48.14% accuracy, far below the human-level accuracy of 98.40%. Extensive experimental analyses are also conducted, suggesting the future research directions. The benchmark and codes are available at https://github.com/ziyan-xiaoyu/SpatialMQA.git.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 19 of its 31 references that are in the map. First posted on arXiv 25 May 2025.

Builds on

19 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  8. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  9. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 128
  10. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 60
  11. 2024CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputscited by 56
  12. 2019SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognitioncited by 55
6 more
  1. 2024A Survey for Foundation Models in Autonomous Drivingcited by 52
  2. 2022Texts as Images in Prompt Tuning for Multi-Label Image Recognitioncited by 51
  3. 2020Rel3D: A Minimally Contrastive Benchmark for Grounding Spatial Relations in 3Dcited by 30
  4. 2023Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learningcited by 30
  5. 2023MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Imagescited by 18
  6. 2024Can Transformers Capture Spatial Relations between Objects?cited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 14.

20 papers.

Sort:

By area:

  1. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
  2. 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 12
  3. 2025Spatial-aware Vision Language Model for Autonomous Drivingcited by 4
  4. 2026TangramPuzzle: Evaluating Multimodal Large Language Models with Compositional Spatial Reasoningcited by 4
  5. 2026The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agentscited by 4
  6. 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
  7. 2026Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligencecited by 3
  8. 2025Can VLMs Predict Future States? Bootstrapping World Models from Inverse Dynamicscited by 2
  9. 2026Seeing Isn't Knowing: Do VLMs Know When Not to Answer Spatial Questions (and Why)?cited by 2
  10. 2025MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlookcited by 1
  11. 2026AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretationcited by 1
  12. 2026PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoningcited by 1
8 more
  1. 2026BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modelingcited by 1
  2. 2025Action Without Interaction: Probing the Physical Foundations of Video LMMs via Contact-Release Detectioncited by 0
  3. 2026On Semiotic-Grounded Interpretive Evaluation of Generative Artcited by 0
  4. 2026Dual-Pathway Geometry-Aware MLLM for Spatial Intelligencecited by 0
  5. 2026TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMscited by 0
  6. 2026UniGeo: A Multi-modal Large Language Model for Text-Guided Cross-View Geo-Localizationcited by 0
  7. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
  8. 2026SpatialCORE: Confidence-Aware Grounded Spatial Reasoning in Large Vision--Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards a Data-Centric Understanding of Spatial Reasoning in Multimodal Large Language Models
  2. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  3. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  4. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligence
  5. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  6. 2025The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Study
  7. 2026Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
  8. 2022Visual Spatial Reasoning