# Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation

arXiv: https://arxiv.org/abs/2504.17207 · HTML: https://mlsanity.com/paper/2504.17207/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 93 references that are in the map.
- First posted on arXiv 24 Apr 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 20 of 20)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V](https://mlsanity.com/paper/2310.11441/) (2023)
- [Imagine while Reasoning in Space: Multimodal Visualization-of-Thought](https://mlsanity.com/paper/2501.07542/) (2025)
- [3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark](https://mlsanity.com/paper/2412.07825/) (2024)
- [SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models](https://mlsanity.com/paper/2412.07755/) (2024)
- [Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings](https://mlsanity.com/paper/2305.02317/) (2023)
- [Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models](https://mlsanity.com/paper/2405.13872/) (2024)
- [SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning](https://mlsanity.com/paper/2501.10074/) (2025)
- [Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models](https://mlsanity.com/paper/2402.12058/) (2024)
- [Visual Prompting in Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2409.15310/) (2024)
- [Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models](https://mlsanity.com/paper/2412.18605/) (2024)
- [Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning](https://mlsanity.com/paper/2410.16162/) (2024)
- [SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation](https://mlsanity.com/paper/2412.12693/) (2024)
- [Open Vocabulary Monocular 3D Object Detection](https://mlsanity.com/paper/2411.16833/) (2024)
- [Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models](https://mlsanity.com/paper/2409.12969/) (2024)

## Built on by (most cited first, 30 of 42)

- [ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models](https://mlsanity.com/paper/2505.21500/) (2025)
- [OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models](https://mlsanity.com/paper/2506.03135/) (2025)
- [Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views](https://mlsanity.com/paper/2510.18632/) (2025)
- [Think3D: Thinking with Space for Spatial Reasoning](https://mlsanity.com/paper/2601.13029/) (2026)
- [Orient Anything V2: Unifying Orientation and Rotation Understanding](https://mlsanity.com/paper/2601.05573/) (2026)
- [SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL](https://mlsanity.com/paper/2512.04069/) (2025)
- [SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs](https://mlsanity.com/paper/2509.25390/) (2025)
- [SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery](https://mlsanity.com/paper/2512.07733/) (2025)
- [Dense360: Dense Understanding from Omnidirectional Panoramas](https://mlsanity.com/paper/2506.14471/) (2025)
- [Abstract 3D Perception for Spatial Intelligence in Vision-Language Models](https://mlsanity.com/paper/2511.10946/) (2025)
- [Gaze Heads: How VLMs Look at What They Describe](https://mlsanity.com/paper/2606.14703/) (2026)
- [EgoVITA: Learning to Plan and Verify for Egocentric Video Reasoning](https://mlsanity.com/paper/2511.18242/) (2025)
- [MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images](https://mlsanity.com/paper/2511.19119/) (2025)
- [Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling](https://mlsanity.com/paper/2512.01821/) (2025)
- [Transductive Visual Programming: Evolving Tool Libraries from Experience for Spatial Reasoning](https://mlsanity.com/paper/2512.20934/) (2025)
- [From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models](https://mlsanity.com/paper/2602.08735/) (2026)
- [Next-Gen CAPTCHAs: Leveraging the Cognitive Gap for Scalable and Diverse GUI-Agent Defense](https://mlsanity.com/paper/2602.09012/) (2026)
- [VIEW2SPACE: Studying Multi-View Visual Reasoning from Sparse Observations](https://mlsanity.com/paper/2603.16506/) (2026)
- [Grounded Misunderstandings in Asymmetric Dialogue: A Perspectivist Annotation Scheme for MapTask](https://mlsanity.com/paper/2511.03718/) (2025)
- [MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representation](https://mlsanity.com/paper/2602.07082/) (2026)
- [Loc3R-VLM: Language-based Localization and 3D Reasoning with Vision-Language Models](https://mlsanity.com/paper/2603.18002/) (2026)
- [Cognitively-Inspired Tokens Overcome Egocentric Bias in Multimodal Models](https://mlsanity.com/paper/2601.16378/) (2026)
- [Predicting Camera Pose from Perspective Descriptions for Spatial Reasoning](https://mlsanity.com/paper/2602.06041/) (2026)
- [Learning Multi-View Spatial Reasoning from Cross-View Relations](https://mlsanity.com/paper/2603.27967/) (2026)
- [Enhancing MLLM Spatial Understanding via Active 3D Scene Exploration for Multi-Perspective Reasoning](https://mlsanity.com/paper/2604.06725/) (2026)
- [VIABLE: A Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation](https://mlsanity.com/paper/2605.31351/) (2026)
- [Skill-3D: Evolving Scene-Aware Skills for Agentic 3D Spatial Reasoning](https://mlsanity.com/paper/2606.07436/) (2026)
- [Reinforcing Dual-Path Reasoning in Spatial Vision Language Models](https://mlsanity.com/paper/2606.17539/) (2026)
- [SpatialQuery: Benchmarking Geometry-Grounded Multi-Instance Spatial Reasoning in Vision-Language Models](https://mlsanity.com/paper/2608.01709/) (2026)
- [CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport](https://mlsanity.com/paper/2608.13226/) (2026)

## Abstract (arXiv)

We present a framework for perspective-aware reasoning in vision-language models (VLMs) through mental imagery simulation. Perspective-taking, the ability to perceive an environment or situation from an alternative viewpoint, is a key benchmark for human-level visual understanding, essential for environmental interaction and collaboration with autonomous agents. Despite advancements in spatial reasoning within VLMs, recent research has shown that modern VLMs significantly lack perspective-aware reasoning capabilities and exhibit a strong bias toward egocentric interpretations. To bridge the gap between VLMs and human perception, we focus on the role of mental imagery, where humans perceive the world through abstracted representations that facilitate perspective shifts. Motivated by this, we propose a framework for perspective-aware reasoning, named Abstract Perspective Change (APC), that effectively leverages vision foundation models, such as object detection, segmentation, and orientation estimation, to construct scene abstractions and enable perspective transformations. Our experiments on synthetic and real-image benchmarks, compared with various VLMs, demonstrate significant improvements in perspective-aware reasoning with our framework, further outperforming fine-tuned spatial reasoning models and novel-view-synthesis-based approaches.