Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Despite the widespread adoption of MLLMs in embodied agents, their capabilities remain largely confined to reactive planning from immediate observations, consistently failing in spatial reasoning across extensive spatiotemporal scales. Cognitive science reveals that Biological Intelligence (BI) thrives on "mental navigation": the strategic construction of spatial representations from experience and the subsequent mental simulation of paths prior to action. To bridge the gap between AI and BI, we introduce Video2Mental, a pioneering benchmark for evaluating the mental navigation capabilities of MLLMs. The task requires constructing hierarchical cognitive maps from long egocentric videos and generating landmark-based path plans step by step, with planning accuracy verified through simulator-based physical interaction. Our benchmarking results reveal that mental navigation capability does not naturally emerge from standard pre-training. Frontier MLLMs struggle profoundly with zero-shot structured spatial representation, and their planning accuracy decays precipitously over extended horizons. To overcome this, we propose NavMind, a reasoning model that internalizes mental navigation using explicit, fine-grained cognitive maps as learnable intermediate representations. Through a difficulty-stratified progressive supervised fine-tuning paradigm, NavMind effectively bridges the gap between raw perception and structured planning. Experiments demonstrate that NavMind achieves superior mental navigation capabilities, significantly outperforming frontier commercial and spatial MLLMs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 33 references that are in the map. First posted on arXiv 23 Mar 2026.
Builds on
23 of 33 references.
Closest ancestors · 1
Same direction, newest first
Other references · 22
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2025OpenAI GPT-5 System Cardcited by 910
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2022Visual Language Maps for Robot Navigationcited by 362
10 more
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 357
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 123
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 91
- 2025RoboBrain 2.0 Technical Reportcited by 87
- 2023Embodied Task Planning with Large Language Modelscited by 84
- 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 75
- 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 61
- 2025MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligencecited by 34
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
- 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
- 2025NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
- 2025SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition
- 2025City Navigation in the Wild: Exploring Emergent Navigation from Web-Scale Knowledge in MLLMs
- 2025Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning
- 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
- 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigation