mlsanityData to 14 Oct 2026

Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework

arXiv 2610.17089 ↗First posted 12 Oct 2026cs.RO cs.AIcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require opening a door to access a room and then opening a container to reveal a hidden target. We formulate interactive navigation beyond reachability, unifying channel-induced changes in spatial connectivity, container-induced changes in target visibility, and their sequential dependencies. We introduce MultINav-Bench, a 3K-episode ObjectGoal benchmark in MolmoSpaces spanning channel, container, and mixed tasks, with interaction-necessity annotations covering required and unnecessary episodes. Its data generation pipeline automates task construction across multiple task types. The evaluation protocol separately measures navigation success, interaction effectiveness, and cost. We further propose MultINav-Agent, a three-stage agentic framework built on a persistent, state-conditioned interaction graph. Semantic interactive perception grounds entities and interaction states; a vision-language model selects task-relevant actions based on expected changes in connectivity and target observability; and post-interaction verification updates the graph to guide subsequent navigation, exploration, or interaction. Experiments on MultINav-Bench show that MultINav-Agent consistently outperforms adapted baselines across channel, container, and mixed tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 50 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

43 of 50 references.

References · 43

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  3. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  7. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
  8. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  9. 2018On Evaluation of Embodied Navigation Agentscited by 547
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  11. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
  12. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
31 more
  1. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  2. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  3. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  4. 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
  5. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
  6. 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 148
  7. 20203D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humanscited by 144
  8. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  9. 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 90
  10. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
  11. 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 70
  12. 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
  13. 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 55
  14. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 52
  15. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 48
  16. 2024Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulationcited by 39
  17. 2024SPIN: Simultaneous Perception, Interaction and Navigationcited by 29
  18. 2023Optimal Scene Graph Planning with Large Language Model Guidancecited by 27
  19. 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 23
  20. 2024SEEK: Semantic Reasoning for Object Goal Navigation in Real World Inspection Taskscited by 19
  21. 2022Sequential Manipulation Planning on Scene Graphcited by 17
  22. 2024UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environmentscited by 17
  23. 2021Pushing it out of the Way: Interactive Visual Navigationcited by 16
  24. 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
  25. 2026MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulationcited by 14
  26. 2024Task and Motion Planning in Hierarchical 3D Scene Graphscited by 12
  27. 2026Articulated 3D Scene Graphs for Open-World Mobile Manipulationcited by 11
  28. 2025MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planningcited by 9
  29. 2026Pandora: Articulated 3D Scene Graphs from Egocentric Visioncited by 6
  30. 2024Interactive Navigation with Adaptive Non-prehensile Mobile Manipulationcited by 1
  31. 2026CoINS: Counterfactual Interactive Navigation via Skill-Aware VLMcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
  2. 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
  3. 2025NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
  4. 2026CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation
  5. 2025NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything
  6. 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigation
  7. 2026NavHarness: Towards Lifelong Embodied Navigation
  8. 2026ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop