Interactive Navigation Beyond Reachability: Benchmark and Agentic Framework
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Existing embodied navigation tasks commonly assume that traversability and object states remain fixed during an episode. Yet reaching a goal may require opening a door to access a room and then opening a container to reveal a hidden target. We formulate interactive navigation beyond reachability, unifying channel-induced changes in spatial connectivity, container-induced changes in target visibility, and their sequential dependencies. We introduce MultINav-Bench, a 3K-episode ObjectGoal benchmark in MolmoSpaces spanning channel, container, and mixed tasks, with interaction-necessity annotations covering required and unnecessary episodes. Its data generation pipeline automates task construction across multiple task types. The evaluation protocol separately measures navigation success, interaction effectiveness, and cost. We further propose MultINav-Agent, a three-stage agentic framework built on a persistent, state-conditioned interaction graph. Semantic interactive perception grounds entities and interaction states; a vision-language model selects task-relevant actions based on expected changes in connectivity and target observability; and post-interaction verification updates the graph to guide subsequent navigation, exploration, or interaction. Experiments on MultINav-Bench show that MultINav-Agent consistently outperforms adapted baselines across channel, container, and mixed tasks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 43 of its 50 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
43 of 50 references.
References · 43
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 750
- 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
- 2018On Evaluation of Embodied Navigation Agentscited by 547
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 429
- 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
31 more
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
- 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
- 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
- 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
- 2024BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulationcited by 148
- 20203D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humanscited by 144
- 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
- 2019Interactive Gibson Benchmark (iGibson 0.5): A Benchmark for Interactive Navigation in Cluttered Environmentscited by 90
- 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
- 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 70
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
- 2019HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulatorscited by 55
- 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 52
- 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 48
- 2024Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulationcited by 39
- 2024SPIN: Simultaneous Perception, Interaction and Navigationcited by 29
- 2023Optimal Scene Graph Planning with Large Language Model Guidancecited by 27
- 2025FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interactioncited by 23
- 2024SEEK: Semantic Reasoning for Object Goal Navigation in Real World Inspection Taskscited by 19
- 2022Sequential Manipulation Planning on Scene Graphcited by 17
- 2024UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environmentscited by 17
- 2021Pushing it out of the Way: Interactive Visual Navigationcited by 16
- 2024The One RING: a Robotic Indoor Navigation Generalistcited by 16
- 2026MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulationcited by 14
- 2024Task and Motion Planning in Hierarchical 3D Scene Graphscited by 12
- 2026Articulated 3D Scene Graphs for Open-World Mobile Manipulationcited by 11
- 2025MomaGraph: State-Aware Unified Scene Graphs with Vision-Language Model for Embodied Task Planningcited by 9
- 2026Pandora: Articulated 3D Scene Graphs from Egocentric Visioncited by 6
- 2024Interactive Navigation with Adaptive Non-prehensile Mobile Manipulationcited by 1
- 2026CoINS: Counterfactual Interactive Navigation via Skill-Aware VLMcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
- 2026HarnessVLN: Unifying Training-Free Embodied Navigation through an Agent Harness
- 2025NavSpace: How Navigation Agents Follow Spatial Intelligence Instructions
- 2026CGFM-Nav: Cognitive Graph-Field Memory for Semantic-Guided Lifelong Multimodal Embodied Navigation
- 2025NavA3: Understanding Any Instruction, Navigating Anywhere, Finding Anything
- 2025Expand Your SCOPE: Semantic Cognition over Potential-Based Exploration for Embodied Visual Navigation
- 2026NavHarness: Towards Lifelong Embodied Navigation
- 2026ESI-Bench: Towards Embodied Spatial Intelligence that Closes the Perception-Action Loop