# SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation

arXiv: https://arxiv.org/abs/2412.12693 · HTML: https://mlsanity.com/paper/2412.12693/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 17 of 17)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities](https://mlsanity.com/paper/2308.02490/) (2023)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [What's "up" with vision-language models? Investigating their struggle with spatial reasoning](https://mlsanity.com/paper/2310.19785/) (2023)
- [Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration](https://mlsanity.com/paper/2306.09093/) (2023)
- [The Generative AI Paradox: "What It Can Create, It May Not Understand"](https://mlsanity.com/paper/2311.00059/) (2023)
- [CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation](https://mlsanity.com/paper/2311.18775/) (2023)
- [SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention](https://mlsanity.com/paper/2312.01990/) (2023)
- [What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases](https://mlsanity.com/paper/2404.02415/) (2024)

## Built on by (most cited first, 30 of 31)

- [ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models](https://mlsanity.com/paper/2505.21500/) (2025)
- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views](https://mlsanity.com/paper/2510.18632/) (2025)
- [How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective](https://mlsanity.com/paper/2509.18905/) (2025)
- [Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation](https://mlsanity.com/paper/2504.17207/) (2025)
- [Spatial Understanding from Videos: Structured Prompts Meet Simulation Data](https://mlsanity.com/paper/2506.03642/) (2025)
- [SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs](https://mlsanity.com/paper/2509.25390/) (2025)
- [Towards Cross-View Point Correspondence in Vision-Language Models](https://mlsanity.com/paper/2512.04686/) (2025)
- [SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery](https://mlsanity.com/paper/2512.07733/) (2025)
- [Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding](https://mlsanity.com/paper/2603.19235/) (2026)
- [VRIQ: Benchmarking and Analyzing Visual-Reasoning IQ of VLMs](https://mlsanity.com/paper/2602.05382/) (2026)
- [GRAID: Enhancing Spatial Reasoning of VLMs Through High-Fidelity Data Generation](https://mlsanity.com/paper/2510.22118/) (2025)
- [Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling](https://mlsanity.com/paper/2512.01821/) (2025)
- [CubeBench: Diagnosing Interactive, Long-Horizon Spatial Reasoning Under Partial Observations](https://mlsanity.com/paper/2512.23328/) (2025)
- [Color Names in Vision-Language Models](https://mlsanity.com/paper/2509.22524/) (2025)
- [Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models](https://mlsanity.com/paper/2603.18002/) (2026)
- [SATURN: Symbolic Spatial Reasoning for Multi-Perspective Grounding](https://mlsanity.com/paper/2606.22694/) (2026)
- [Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text](https://mlsanity.com/paper/2607.21072/) (2026)
- [Cognitively-Inspired Tokens Overcome Egocentric Bias in Multimodal Models](https://mlsanity.com/paper/2601.16378/) (2026)
- [Order Is Not Layout: Order-to-Space Bias in Image Generation](https://mlsanity.com/paper/2603.03714/) (2026)
- [ViewFusion: Structured Spatial Thinking Chains for Multi-View Reasoning](https://mlsanity.com/paper/2603.06024/) (2026)
- [How Do LLMs and VLMs Understand Viewpoint Rotation Without Vision? An Interpretability Study](https://mlsanity.com/paper/2604.15294/) (2026)
- [PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning](https://mlsanity.com/paper/2605.19382/) (2026)
- [SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes](https://mlsanity.com/paper/2605.31148/) (2026)
- [SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video](https://mlsanity.com/paper/2607.01784/) (2026)
- [Token Warping Helps MLLMs Look from Nearby Viewpoints](https://mlsanity.com/paper/2604.02870/) (2026)
- [AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience](https://mlsanity.com/paper/2606.31467/) (2026)
- [Evaluation of Multilingual Ability to Use Spatial Deictic Expressions in Vision-Language Models](https://mlsanity.com/paper/2607.07251/) (2026)
- [LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learning](https://mlsanity.com/paper/2607.17243/) (2026)
- [AlloEgo-VLM: Disambiguating Allocentric and Egocentric Reference Frames in Vision-Language Models](https://mlsanity.com/paper/2608.15605/) (2026)

## Abstract (arXiv)

Current vision-language models may grasp basic spatial cues and simple directions (e.g. left, right, front, back), but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications. To address this gap, we develop SPHERE (Spatial Perception and Hierarchical Evaluation of REasoning), a hierarchical evaluation framework supported by a new human-annotated dataset. SPHERE systematically probes models across increasing levels of complexity, from fundamental skills to multi-skill integration and high-level reasoning that combines spatial, visual, and logical understanding. Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity, understanding both egocentric and allocentric perspectives, and applying spatial logic in physical contexts. These findings expose critical blind spots in existing models and underscore the need for more advanced spatial reasoning techniques, driving the development of vision-language models that align more closely with human spatial cognition. The SPHERE benchmark is available at https://github.com/zwenyu/SPHERE-VLM.