mlsanityData to 7 Oct 2026

SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation

arXiv 2412.12693 ↗First posted 17 Dec 2024cs.CV cs.AIcited by 31

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Current vision-language models may grasp basic spatial cues and simple directions (e.g. left, right, front, back), but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications. To address this gap, we develop SPHERE (Spatial Perception and Hierarchical Evaluation of REasoning), a hierarchical evaluation framework supported by a new human-annotated dataset. SPHERE systematically probes models across increasing levels of complexity, from fundamental skills to multi-skill integration and high-level reasoning that combines spatial, visual, and logical understanding. Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity, understanding both egocentric and allocentric perspectives, and applying spatial logic in physical contexts. These findings expose critical blind spots in existing models and underscore the need for more advanced spatial reasoning techniques, driving the development of vision-language models that align more closely with human spatial cognition. The SPHERE benchmark is available at https://github.com/zwenyu/SPHERE-VLM.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 33 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

17 of 33 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan
  2. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct

Other references · 15

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  10. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  11. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 168
  12. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 97
3 more
  1. 2023CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generationcited by 63
  2. 2023SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attentioncited by 20
  3. 2024What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biasescited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 18.

31 papers.

Sort:

By area:

  1. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 110
  2. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  3. 2025Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Viewscited by 58
  4. 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulationcited by 41
  5. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  6. 2025Spatial Understanding from Videos: Structured Prompts Meet Simulation Datacited by 28
  7. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMscited by 12
  8. 2025Towards Cross-View Point Correspondence in Vision-Language Modelscited by 12
  9. 2025SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagerycited by 10
  10. 2026Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understandingcited by 6
  11. 2026VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMscited by 4
  12. 2025GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generationcited by 3
19 more
  1. 2025Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modelingcited by 3
  2. 2025CubeBench: Diagnosing Interactive, Long-Horizon Spatial Reasoning Under Partial Observationscited by 3
  3. 2025Color Names in Vision-Language Modelscited by 2
  4. 2026Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Modelscited by 2
  5. 2026SATURN: Symbolic Spatial Reasoning for Multi-Perspective Groundingcited by 2
  6. 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
  7. 2026Cognitively-Inspired Tokens Overcome Egocentric Bias in Multimodal Modelscited by 1
  8. 2026Order Is Not Layout: Order-to-Space Bias in Image Generationcited by 1
  9. 2026ViewFusion: Structured Spatial Thinking Chains for Multi-View Reasoningcited by 1
  10. 2026How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Studycited by 1
  11. 2026PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoningcited by 1
  12. 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
  13. 2026SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Videocited by 1
  14. 2026Token Warping Helps MLLMs Look from Nearby Viewpointscited by 0
  15. 2026AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neurosciencecited by 0
  16. 2026Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Modelscited by 0
  17. 2026LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learningcited by 0
  18. 2026AlloEgo-VLM: Disambiguating Allocentric and Egocentric Reference Frames in Vision-Language Modelscited by 0
  19. 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  2. 2026SpatiaLab: Can Vision-Language Models Perform Spatial Reasoning in the Wild?
  3. 2025Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models
  4. 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
  5. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models
  6. 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models
  7. 2025Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models
  8. 2026Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests