# Beyond Localization: A Comprehensive Diagnosis of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Images

arXiv: https://arxiv.org/abs/2605.12413 · HTML: https://mlsanity.com/paper/2605.12413/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 12 May 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models](https://mlsanity.com/paper/2505.21500/) (2025)
- [MindCube: Spatial Mental Modeling from Limited Views](https://mlsanity.com/paper/2506.21458/) (2025)
- [ShapeWorld - A new test methodology for multimodal language understanding](https://mlsanity.com/paper/1704.04517/) (2017)
- [Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks](https://mlsanity.com/paper/2510.25760/) (2025)
- [Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models](https://mlsanity.com/paper/2409.12969/) (2024)
- [Towards Omnidirectional Reasoning with 360-R1: A Dataset, Benchmark, and GRPO-based Method](https://mlsanity.com/paper/2505.14197/) (2025)
- [Vision Language Models See What You Want but not What You See](https://mlsanity.com/paper/2410.00324/) (2024)
- [Video-MSR: Benchmarking Multi-hop Spatial Reasoning Capabilities of MLLMs](https://mlsanity.com/paper/2601.09430/) (2026)
- [Egocentric Bias in Vision-Language Models](https://mlsanity.com/paper/2602.15892/) (2026)
- [Mitigating the Reasoning Tax in Vision-Language Fine-Tuning with Input-Adaptive Depth Aggregation](https://mlsanity.com/paper/2603.26330/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) show strong visual perception, yet remain limited in reasoning about space under changing viewpoints. We study this challenge as Perspective-Conditioned Spatial Reasoning (PCSR) in 360 degree omnidirectional images, where broad scene coverage reduces ambiguity from partial observations without eliminating the need for viewpoint-dependent inference. To assess this capability, we introduce PCSR-Bench, a diagnostic benchmark of 84,373 QA pairs from 2,600 omnidirectional images across 26 indoor environments, organized into eight tasks under three cognitive groups--- Perception, Spatial, and advanced PCSR. We evaluate 14 representative MLLMs and observe a substantial perception--reasoning gap: accuracy reaches 57.59% on Limited Field-of-View Reasoning (T7) but drops to 13.49%, 7.13%, and 0.64% on Relative Direction (T2), Egocentric Rotation (T4), and open-ended Compositional Directional Chains (T3), respectively. To probe the plasticity of this gap, we conduct an RL-based diagnostic study on a 7B-scale model. Reward shaping improves a matched 7B baseline from 31.10% to 60.06% under a controlled setting, suggesting that PCSR exhibits partial plasticity rather than being fully immutable. Still, these gains are task-selective, sensitive to reward design, and partially dependent on the evaluation protocol. These results position PCSR as a key bottleneck in current MLLMs and highlight meaningful yet bounded room for recovery under targeted optimization. Details and access are available at https://github.com/Caleb-ychen/PCSR-Benchmark.