Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?
Not on the map. Why?
Abstract (arXiv)
Spatial embodied intelligence requires agents to act to acquire information under partial observability. While multimodal foundation models excel at passive perception, their capacity for active, self-directed exploration remains understudied. We propose Theory of Space, defined as an agent's ability to actively acquire information through self-directed, active exploration and to construct, revise, and exploit a spatial belief from sequential, partial observations. We evaluate this through a benchmark where the goal is curiosity-driven exploration to build an accurate cognitive map. A key innovation is spatial belief probing, which prompts models to reveal their internal spatial representations at each step. Our evaluation of state-of-the-art models reveals several critical bottlenecks. First, we identify an Active-Passive Gap, where performance drops significantly when agents must autonomously gather information. Second, we find high inefficiency, as models explore unsystematically compared to program-based proxies. Through belief probing, we diagnose that while perception is an initial bottleneck, global beliefs suffer from instability that causes spatial knowledge to degrade over time. Finally, using a false belief paradigm, we uncover Belief Inertia, where agents fail to update obsolete priors with new evidence. This issue is present in text-based agents but is particularly severe in vision-based models. Our findings suggest that current foundation models struggle to maintain coherent, revisable spatial beliefs during active exploration.
Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 39 of its 66 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
39 of 66 references.
References · 39
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
- 2018VirtualHome: Simulating Household Activities via Programscited by 411
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
- 2017Embodied Question Answeringcited by 335
27 more
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
- 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 245
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
- 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 169
- 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
- 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
- 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
- 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
- 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
- 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 75
- 2024Explore until Confident: Efficient Exploration for Embodied Question Answeringcited by 74
- 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 65
- 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
- 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 62
- 2024Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Modelscited by 54
- 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
- 2025VLM4D: Towards Spatiotemporal Awareness in Vision Language Modelscited by 43
- 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 31
- 2025InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Modelscited by 26
- 2024Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoningcited by 16
- 2024ReALFRED: An Embodied Instruction Following Benchmark in Photo-Realistic Environmentscited by 16
- 2025Enter the Mind Palace: Reasoning and Planning for Long-term Active Embodied Question Answeringcited by 12
Built on by
Citing papers per year: 2026: 18.
18 papers.
Sort:
By area:
- 2026Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agentscited by 7
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
- 2026The Limits of Learning from Pictures and Text: Vision-Language Models and Embodied Scene Understandingcited by 2
- 2026Exploration and Exploitation Errors Are Measurable for Language Model Agentscited by 2
- 20263D Primitives are a Spatial Language for VLMscited by 2
- 2026Planning with the Viewscited by 2
- 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Modelscited by 2
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMscited by 1
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answeringcited by 1
- 2026From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISPcited by 1
- 2026Theory of Code Space: Do Code Agents Understand Software Architecture?cited by 0
6 more
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 0
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMscited by 0
- 2026Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?cited by 0
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Executioncited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning28 Sep 2026
- 2026DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution25 Sep 2026
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control17 Sep 2026
- 2026Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?8 Aug 2026
- 2026From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP25 Jun 2026
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs21 Jun 2026
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering9 Jun 2026
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?31 May 2026
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes29 May 2026
- 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Models28 May 2026
- 2026Planning with the Views28 May 2026
- 20263D Primitives are a Spatial Language for VLMs12 May 2026
6 more
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMs10 May 2026
- 2026Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents9 May 2026
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments24 Apr 2026
- 2026Exploration and Exploitation Errors Are Measurable for Language Model Agents14 Apr 2026
- 2026The Limits of Learning from Pictures and Text: Vision-Language Models and Embodied Scene Understanding27 Mar 2026
- 2026Theory of Code Space: Do Code Agents Understand Software Architecture?28 Feb 2026
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
- 2026From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISPcited by 1
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning28 Sep 2026
- 2026From Hallucination to Grounding: Diagnosing Visual Spatial Intelligence via CRISP25 Jun 2026
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?31 May 2026
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes29 May 2026
- 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments24 Apr 2026
- 2026The Limits of Learning from Pictures and Text: Vision-Language Models and Embodied Scene Understandingcited by 2
- 20263D Primitives are a Spatial Language for VLMscited by 2
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMscited by 1
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMscited by 0
- 2026Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?cited by 0
- 2026Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?8 Aug 2026
- 2026OmniSpace: Efficient Geometry Awareness for Autonomous Vehicles MLLMs21 Jun 2026
- 20263D Primitives are a Spatial Language for VLMs12 May 2026
- 2026SpaceMind++: Toward Allocentric Cognitive Maps for Spatially Grounded Video MLLMs10 May 2026
- 2026The Limits of Learning from Pictures and Text: Vision-Language Models and Embodied Scene Understanding27 Mar 2026
- 2026Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agentscited by 7
- 2026Planning with the Viewscited by 2
- 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Modelscited by 2
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answeringcited by 1
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control17 Sep 2026
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering9 Jun 2026
- 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Models28 May 2026
- 2026Planning with the Views28 May 2026
- 2026Done, But Not Sure: Disentangling World Completion from Self-Termination in Embodied Agents9 May 2026