mlsanityData to 13 Oct 2026

ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop

arXiv 2605.18746 ↗First posted 18 May 2026cs.CV cs.AI cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Spatial intelligence unfolds through a perception-action loop: agents act to acquire observations, and reason about how observations vary as a function of action. Rather than passively processing what is seen, they actively uncover what is unseen - occluded structure, dynamics, containment, and functionality that cannot be resolved from passive sensing alone. We move beyond prior formulations of spatial intelligence that assume oracle observations by recasting the observer as an actor. We introduce ESI-BENCH, a comprehensive benchmark for embodied spatial intelligence spanning 10 task categories and 29 subcategories built on OmniGibson, grounded in Spelke's core knowledge systems. Agents must decide what abilities to deploy - perception, locomotion, and manipulation - and how to sequence them to actively accumulate task-relevant evidence. We conduct extensive experiments on state-of-the-art MLLMs and find that active exploration substantially outperforms passive counterparts, with agents spontaneously discovering emergent spatial strategies without explicit instructions, while random multi-view often adds noise rather than signal despite consuming far more images. Most failures stem not from weak perception but from action blindness: poor action choices lead to poor observations, which in turn drive cascading errors. While explicit 3D grounding stabilizes reasoning on depth-sensitive tasks, imperfect 3D representation proves more harmful than 2D baselines by distorting spatial relations. Human studies further reveal that unlike humans who seek falsifying viewpoints and revise beliefs under contradiction, models commit prematurely with high confidence regardless of evidence quality, exposing a metacognitive gap that neither better perception nor more embodied interaction alone can close.

Key facts. Cited by 20 papers in the map (data through 13 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

17 of 27 references.

References · 17

  1. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 687
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 588
  3. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 217
  4. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 209
  5. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 172
  6. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 157
  7. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 148
  8. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 135
  9. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 132
  10. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
  11. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 56
  12. 2025Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answeringcited by 29
5 more
  1. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 16
  2. 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 14
  3. 2026CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoningcited by 7
  4. 2026From Perception to Action: An Interactive Benchmark for Vision Reasoningcited by 4
  5. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Modelscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026Self-Evolving Embodied Agents via Skill-Harness Evolutioncited by 9
  2. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 4
  3. 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulationcited by 2
  4. 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 2
  5. 2026SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessitycited by 1
  6. 2026SpatialUAV: Benchmarking Spatial Intelligence for Low-Altitude UAV Perception, Collaboration, and Motioncited by 1
  7. 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
  8. 2026Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?cited by 0
  9. 2026PROBE: Manipulation-Grounded Visual Question Answering with VLM Agentscited by 0
  10. 2026ParallelWorld: Test-Time Scaling for Embodied Reasoningcited by 0
  11. 2026MindTopo: Can Foundation Models Reason in Topological Space?cited by 0
  12. 2026Space as an Interventional Invariant: Cross-Modal Predictive Geometry for Stratified Cities and Em-Spaced Intelligencecited by 0
8 more
  1. 2026VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Controlcited by 0
  2. 2026ActiveArena: Benchmarking and Understanding Active Perception in Robotic Manipulationcited by 0
  3. 2026Beyond Spatial Benchmarks: From Spatial Reasoning to Navigationcited by 0
  4. 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoningcited by 0
  5. 2026VCN-Bench: A Video-Contextualized Navigation Benchmark for Spatial Reasoning over Prior Visual Experiencecited by 0
  6. 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0
  7. 2026ROMA: LLM System for Real-World Object-Centric Multi-Sensory Active Perceptioncited by 0
  8. 2026Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the Wildcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Perception to Action: Spatial AI Agents and World Models
  2. 2026RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents
  3. 2025From reactive to cognitive: brain-inspired spatial intelligence for embodied agents
  4. 2026Vision-language models lag human performance on physical dynamics and intent reasoning
  5. 2025Mind Meets Space: Rethinking Agentic Spatial Intelligence from a Neuroscience-inspired Perspective
  6. 2025Embodied World Models Emerge from Navigational Task in Open-Ended Environments
  7. 2026Embodied-BenchClaw: An Autonomous Multi-Agent System for Embodied Spatial Intelligence Benchmark Construction
  8. 2026ACE-Brain-0.5: A Unified Embodied Foundational Model for Physical Agentic AI