ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop
Not on the map. Why?
Abstract (arXiv)
Spatial intelligence unfolds through a perception-action loop: agents act to acquire observations, and reason about how observations vary as a function of action. Rather than passively processing what is seen, they actively uncover what is unseen - occluded structure, dynamics, containment, and functionality that cannot be resolved from passive sensing alone. We move beyond prior formulations of spatial intelligence that assume oracle observations by recasting the observer as an actor. We introduce ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson, grounded in Spelke's core knowledge systems. Agents must decide what abilities to deploy - perception, locomotion, and manipulation - and how to sequence them to actively accumulate task-relevant evidence. We conduct extensive experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts, with agents spontaneously discovering emergent spatial strategies without explicit instructions, while random multi-view often adds noise rather than signal despite consuming far more images. Most failures stem not from weak perception but from action blindness: poor action choices lead to poor observations, which in turn drive cascading errors. While explicit 3D grounding stabilizes reasoning on depth-sensitive tasks, imperfect 3D representation proves more harmful than 2D baselines by distorting spatial relations. Human studies further reveal that unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality, exposing a metacognitive gap that neither better perception nor more embodied interaction alone can close.
Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
17 of 27 references.
References · 17
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 687
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 588
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 217
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 209
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 172
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 157
- 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 148
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 135
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 132
- 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
- 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 56
- 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
5 more
- 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 16
- 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 14
- 2026CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoningcited by 7
- 2026From Perception to Action: An Interactive Benchmark for Vision Reasoningcited by 4
- 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Modelscited by 3
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026Self-Evolving Embodied Agents via Skill-Harness Evolutioncited by 9
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 4
- 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulationcited by 2
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 2
- 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessitycited by 1
- 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motioncited by 1
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 0
- 2026PROBE: Manipulation-Grounded Visual Question Answering with VLM Agentscited by 0
- 2026ParallelWorld: Test-Time Scaling for Embodied Reasoningcited by 0
- 2026MindTopo: Can Foundation Models Reason in Topological Space?cited by 0
- 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligencecited by 0
8 more
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
- 2026ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulationcited by 0
- 2026Beyond Spatial Benchmarks: From Spatial Reasoning to Navigationcited by 0
- 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoningcited by 0
- 2026VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experiencecited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
- 2026ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perceptioncited by 0
- 2026Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wildcited by 0
- 2026Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wild8 Oct 2026
- 2026ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception3 Oct 2026
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning28 Sep 2026
- 2026VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience28 Sep 2026
- 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning28 Sep 2026
- 2026Beyond Spatial Benchmarks: From Spatial Reasoning to Navigation24 Sep 2026
- 2026ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulation21 Sep 2026
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control17 Sep 2026
- 2026MindTopo: Can Foundation Models Reason in Topological Space?10 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026ParallelWorld: Test-Time Scaling for Embodied Reasoning24 Aug 2026
- 2026PROBE: Manipulation-Grounded Visual Question Answering with VLM Agents17 Aug 2026
8 more
- 2026Self-Evolving Embodied Agents via Skill-Harness Evolution11 Aug 2026
- 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligence10 Aug 2026
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?29 Jul 2026
- 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion26 Jun 2026
- 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation24 Jun 2026
- 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity24 Jun 2026
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs2 Jun 2026
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?31 May 2026
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 0
- 2026PROBE: Manipulation-Grounded Visual Question Answering with VLM Agentscited by 0
- 2026ParallelWorld: Test-Time Scaling for Embodied Reasoningcited by 0
- 2026MindTopo: Can Foundation Models Reason in Topological Space?cited by 0
- 2026Beyond Spatial Benchmarks: From Spatial Reasoning to Navigationcited by 0
- 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoningcited by 0
- 2026VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experiencecited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning28 Sep 2026
- 2026VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experience28 Sep 2026
- 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning28 Sep 2026
- 2026Beyond Spatial Benchmarks: From Spatial Reasoning to Navigation24 Sep 2026
- 2026MindTopo: Can Foundation Models Reason in Topological Space?10 Sep 2026
- 2026ParallelWorld: Test-Time Scaling for Embodied Reasoning24 Aug 2026
- 2026PROBE: Manipulation-Grounded Visual Question Answering with VLM Agents17 Aug 2026
- 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?31 May 2026
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 4
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 2
- 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessitycited by 1
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?29 Jul 2026
- 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity24 Jun 2026
- 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMs2 Jun 2026
- 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulationcited by 2
- 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motioncited by 1
- 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligencecited by 0
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
- 2026ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulationcited by 0
- 2026ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perceptioncited by 0
- 2026ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perception3 Oct 2026
- 2026ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulation21 Sep 2026
- 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control17 Sep 2026
- 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligence10 Aug 2026
- 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motion26 Jun 2026
- 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation24 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2026From Perception to Action: Spatial AI Agents and World Models
- 2026RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents
- 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
- 2026Vision-language models lag human performance on physical dynamics and intent reasoning
- 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective
- 2025Embodied World Models Emerge from Navigational Task in Open-Ended Environments
- 2026Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
- 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI