Beyond Localization: A Comprehensive Diagnosis of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Multimodal Large Language Models (MLLMs) show strong visual perception, yet remain limited in reasoning about space under changing viewpoints. We study this challenge as Perspective-Conditioned Spatial Reasoning (PCSR) in 360 degree omnidirectional images, where broad scene coverage reduces ambiguity from partial observations without eliminating the need for viewpoint-dependent inference. To assess this capability, we introduce PCSR-Bench, a diagnostic benchmark of 84,373 QA pairs from 2,600 omnidirectional images across 26 indoor environments, organized into eight tasks under three cognitive groups--- Perception, Spatial, and advanced PCSR. We evaluate 14 representative MLLMs and observe a substantial perception--reasoning gap: accuracy reaches 57.59% on Limited Field-of-View Reasoning (T7) but drops to 13.49%, 7.13%, and 0.64% on Relative Direction (T2), Egocentric Rotation (T4), and open-ended Compositional Directional Chains (T3), respectively. To probe the plasticity of this gap, we conduct an RL-based diagnostic study on a 7B-scale model. Reward shaping improves a matched 7B baseline from 31.10% to 60.06% under a controlled setting, suggesting that PCSR exhibits partial plasticity rather than being fully immutable. Still, these gains are task-selective, sensitive to reward design, and partially dependent on the evaluation protocol. These results position PCSR as a key bottleneck in current MLLMs and highlight meaningful yet bounded room for recovery under targeted optimization. Details and access are available at https://github.com/Caleb-ychen/PCSR-Benchmark.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 54 references that are in the map. First posted on arXiv 12 May 2026.
Builds on
16 of 54 references.
Closest ancestors · 4
Same direction, newest first
- 2026Video-MSR: Benchmarking Multi-hop Spatial Reasoning Capabilities of MLLMs14 Jan
- 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks29 Oct
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models27 May
- 2025Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Method20 May
Other references · 12
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Let's Verify Step by Stepcited by 3,598
- 2015VQA: Visual Question Answeringcited by 2,489
- 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
- 2017ShapeWorld - A new test methodology for multimodal language understandingcited by 37
- 2024Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Modelscited by 15
- 2024Vision Language Models See What You Want but not What You Seecited by 10
- 2026Egocentric Bias in Vision-Language Modelscited by 3
- 2026Mitigating the Reasoning Tax in Vision-Language Fine-Tuning with Input-Adaptive Depth Aggregationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?
- 2026MindEdit-Bench: Benchmarking Object-Level Counterfactual Spatial Reasoning in VLMs from In-the-Wild Photos
- 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
- 2026OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
- 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
- 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models
- 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective
- 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes