mlsanityData to 14 Oct 2026

The Spatial Blindspot of Vision-Language Models

arXiv 2601.09954 ↗First posted 15 Jan 2026cs.CVcited by 8

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision-language models (VLMs) have advanced rapidly, but their ability to capture spatial relationships remains a blindspot. Current VLMs are typically built with contrastive language-image pretraining (CLIP) style image encoders. The training recipe often flattens images into 1D patch sequences, discarding the 2D structure necessary for spatial reasoning. We argue that this lack of spatial awareness is a missing dimension in VLM design and a bottleneck for applications requiring spatial grounding, such as robotics and embodied AI. To address this, we investigate (i) image encoders trained with alternative objectives and (ii) 2D positional encodings. Our experiments show that these architectural choices can lead to improved spatial reasoning on several benchmarks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 40 references that are in the map. First posted on arXiv 15 Jan 2026.

Builds on

31 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Why Do MLLMs Struggle with Spatial Understanding? A Systematic Analysis from Data to Architecture2 Sep
  2. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes21 Apr
  3. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasoners4 Jun
  4. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  11. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  12. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
15 more
  1. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
  2. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  3. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  4. 2022Visual Spatial Reasoningcited by 293
  5. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 255
  6. 2025SmolVLM: Redefining small and efficient multimodal modelscited by 237
  7. 2024MM1: Methods, Analysis & Insights from Multimodal LLM Pre-trainingcited by 232
  8. 2018TallyQA: Answering Complex Counting Questionscited by 153
  9. 2024Multimodal Autoregressive Pre-training of Large Vision Encoderscited by 109
  10. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  11. 2023SILC: Improving Vision Language Pretraining with Self-Distillationcited by 66
  12. 2024Visual Style Prompting with Swapping Self-Attentioncited by 63
  13. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  14. 2025LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuningcited by 28
  15. 2025On the Limitations of Vision-Language Models in Understanding Image Transformscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
  2. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  3. 20263D Primitives are a Spatial Language for VLMscited by 2
  4. 2026Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Interventioncited by 2
  5. 2026Order Is Not Layout: Order-to-Space Bias in Image Generationcited by 1
  6. 2026The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoningcited by 0
  7. 2026Readout Blindness: VLM Scores Miss the Spatial Direction Their Frozen Encoders Retaincited by 0
  8. 2026How Vision Transformers Represent Position: A Mechanistic Comparison of Absolute and Rotary Positional Encodingscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA
  2. 2024SpatialBot: Precise Spatial Understanding with Vision Language Models
  3. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
  4. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  5. 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data
  6. 2026Chain of Spatial Thoughts: Modality-Agnostic Spatial Grounding for Vision Language Models
  7. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
  8. 2026SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models