mlsanityData to 7 Oct 2026

Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models

arXiv 2406.14852 ↗First posted 21 Jun 2024cs.CV cs.AIcited by 111

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

Large language models (LLMs) and vision-language models (VLMs) have demonstrated remarkable performance across a wide range of tasks and domains. Despite this promise, spatial understanding and reasoning -- a fundamental component of human cognition -- remains under-explored. We propose SpatialEval, a novel benchmark that covers diverse aspects of spatial reasoning such as relationship understanding, navigation, and counting. We conduct a comprehensive evaluation of competitive language and vision-language models. Our findings reveal several counter-intuitive insights that have been overlooked in the literature: (1) Spatial reasoning poses significant challenges where competitive models can fall behind random guessing; (2) Despite additional visual input, VLMs often under-perform compared to their LLM counterparts; (3) When both textual and visual information is available, multi-modal language models become less reliant on visual information if sufficient textual clues are provided. Additionally, we demonstrate that leveraging redundancy between vision and text can significantly enhance model performance. We hope our study will inform the development of multimodal models to improve spatial intelligence and further close the gap with human intelligence.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 77 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

34 of 77 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning19 Dec

Other references · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2023Mistral 7Bcited by 3,065
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
21 more
  1. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  2. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  3. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  4. 2022In-context Learning and Induction Headscited by 930
  5. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  6. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  7. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  8. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  9. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  11. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
  12. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 230
  13. 2024Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviewscited by 160
  14. 2024Efficient Multimodal Learning from Data-centric Perspectivecited by 120
  15. 2021Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Colorcited by 118
  16. 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 75
  17. 2024Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Datacited by 61
  18. 2023A Short Survey of Viewing Large Language Models in Legal Aspectcited by 51
  19. 2024IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representationscited by 37
  20. 2024Fourier Circuits in Neural Networks and Transformers: A Case Study of Modular Arithmetic with Multiple Inputscited by 22
  21. 2024Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 58; 2026: 33.

111 papers.

Sort:

By area:

  1. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  2. 2025Phi-4-reasoning Technical Reportcited by 146
  3. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  4. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  5. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 130
  6. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
  7. 2024Process Reward Model with Q-Value Rankingscited by 78
  8. 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
  9. 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 62
  10. 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMscited by 48
  11. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Aheadcited by 43
  12. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
38 more
  1. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalabilitycited by 39
  2. 2025Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learningcited by 38
  3. 2025Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMscited by 35
  4. 2024Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoningcited by 33
  5. 2025Introducing Visual Perception Token into Multimodal Large Language Modelcited by 26
  6. 2025SITE: towards Spatial Intelligence Thorough Evaluationcited by 26
  7. 2025iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMscited by 24
  8. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 22
  9. 2024Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Timecited by 22
  10. 2025Mind with Eyes: from Language Reasoning to Multimodal Reasoningcited by 22
  11. 2025Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?cited by 22
  12. 2024Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Modelcited by 21
  13. 2024The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexitycited by 21
  14. 2024Looped ReLU MLPs May Be All You Need as Practical Programmable Computerscited by 19
  15. 2024HSR-Enhanced Sparse Attention Accelerationcited by 19
  16. 2025SpatialScore: Towards Comprehensive Evaluation for Spatial Intelligencecited by 19
  17. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  18. 2024Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Modelscited by 17
  19. 2024Provable Differentially Private Computation of the Cross-Attention Mechanismcited by 14
  20. 2026pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoningcited by 12
  21. 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 11
  22. 2026IVC-Prune: Revealing the Implicit Visual Coordinates in LVLMs for Vision Token Pruningcited by 11
  23. 2025Circuit Complexity Bounds for Visual Autoregressive Modelcited by 10
  24. 2025From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Taskcited by 10
  25. 2025AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learningcited by 10
  26. 2025Benchmarking and Confidence Evaluation of LALMs For Temporal Reasoningcited by 10
  27. 2025Theoretical Guarantees for High Order Trajectory Refinement in Generative Flowscited by 9
  28. 2025Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specializationcited by 9
  29. 2025Benchmarking LLMs' Swarm intelligencecited by 8
  30. 2025Reinforcement Learning for Large Model: A Surveycited by 8
  31. 2025PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehuntscited by 7
  32. 2026DatBench: Discriminative, Faithful, and Efficient VLM Evaluationscited by 7
  33. 2024ROOT: VLM based System for Indoor Scene Understanding and Beyondcited by 6
  34. 2025PairBench: Are Vision-Language Models Reliable at Comparing What They See?cited by 6
  35. 2025Synthetic Visual Genomecited by 6
  36. 2025SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Modelscited by 6
  37. 2025Visually Prompted Benchmarks Are Surprisingly Fragilecited by 6
  38. 2024Beyond Captioning: Task-Specific Prompting for Improved VLM Performance in Mathematical Reasoningcited by 5

50 most cited of 111 All 111 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  2. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  3. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  4. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  5. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  6. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
  7. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
  8. 2026Can LLMs See Without Pixels? Benchmarking Spatial Intelligence from Textual Descriptions