From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Spatial cognition enables adaptive goal-directed behavior by constructing internal models of space. Robust biological systems consolidate spatial knowledge into three interconnected forms: textitlandmarks for salient cues, textitroute knowledge for movement trajectories, and textitsurvey knowledge for map-like representations. While recent advances in multi-modal large language models (MLLMs) have enabled visual-language reasoning in embodied agents, these efforts lack structured spatial memory and instead operate reactively, limiting their generalization and adaptability in complex real-world environments. Here we present Brain-inspired Spatial Cognition for Navigation (BSC-Nav), a unified framework for constructing and leveraging structured spatial memory in embodied agents. BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals. Integrated with powerful MLLMs, BSC-Nav achieves state-of-the-art efficacy and efficiency across diverse navigation tasks, demonstrates strong zero-shot generalization, and supports versatile embodied behaviors in the real physical world, offering a scalable and biologically grounded path toward general-purpose spatial intelligence.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 74 references that are in the map. First posted on arXiv 24 Aug 2025.
Builds on
43 of 74 references.
References · 43
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
31 more
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 503
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
- 2022Visual Language Maps for Robot Navigationcited by 365
- 2017Embodied Question Answeringcited by 336
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2023ViNT: A Foundation Model for Visual Navigationcited by 222
- 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
- 2021Embodied Intelligence via Learning and Evolutioncited by 163
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
- 2022Navigating to Objects in the Real Worldcited by 106
- 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
- 2023GOAT: GO to Any Thingcited by 89
- 2023Embodied Task Planning with Large Language Modelscited by 84
- 2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
- 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 77
- 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
- 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
- 2024Does Spatial Cognition Emerge in Frontier Models?cited by 66
- 2025UniGoal: Towards Universal Zero-shot Goal-oriented Navigationcited by 57
- 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 55
- 2023Navigating to Objects Specified by Imagescited by 51
- 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 31
- 2022Offline Reinforcement Learning for Visual Navigationcited by 29
- 2024A systematic review of geospatial location embedding approaches in large language models: A path to spatial AI systemscited by 6
Built on by
Citing papers per year: 2025: 4; 2026: 3.
7 papers.
- 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Modelscited by 16
- 2025EchoVLA: Robotic Vision-Language-Action Model with Synergistic Declarative Memory for Mobile Manipulationcited by 15
- 2025Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactionscited by 6
- 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoningcited by 4
- 2025MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memorycited by 3
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMscited by 1
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulationcited by 0
- 2026VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation14 Jul 2026
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMs10 May 2026
- 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models28 Jan 2026
- 2025MG-Nav: Dual-Scale Visual Navigation via Sparse Spatial Memory27 Nov 2025
- 2025EchoVLA: Robotic Vision-Language-Action Model with Synergistic Declarative Memory for Mobile Manipulation22 Nov 2025
- 2025Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions20 Nov 2025
- 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoning20 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective
- 2026Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
- 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
- 2025Embodied World Models Emerge from Navigational Task in Open-Ended Environments
- 2025NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
- 2025Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigation
- 2025A Survey of Large Language Model-Powered Spatial Intelligence Across Scales: Advances in Embodied Agents, Smart Cities, and Earth Science
- 2026CoCoSI: Collaborative Cognitive Map Construction for Spatial Intelligence