mlsanityData to 7 Oct 2026

Ask When It Pays: Cost-Aware Open-Ended Interaction for Instance Goal Navigation

arXiv 2606.03175 ↗First posted 2 Jun 2026cs.CV cs.ROcited by 2

Not on the map. Why?

Abstract (arXiv)

Instance Goal Navigation (IGN) requires an embodied agent to find a specific object instance among distractors from an under-specified natural-language description. Such ambiguity often cannot be resolved from perception and language alone, making interaction with an oracle a natural mechanism for disambiguation. Prior interactive methods allow oracle queries but treat lightweight clarification and route-level guidance alike, letting agents boost success rate through repeated high-information questions rather than by resolving the underlying ambiguity efficiently. We recast interactive IGN as a cost-sensitive uncertainty-reduction problem, where the agent should ask the question whose answer provides the largest reduction in navigation uncertainty relative to its penalty. To this end, we apply an information-gain analysis on existing navigation corpora to identify which cues reduce navigation uncertainty, yielding a compact set of question types and data-derived weights. However, existing interactive navigation benchmarks do not model the cost of different question types or evaluate how efficiently agents use interaction, making them unsuitable for studying cost-sensitive interaction. Based on this taxonomy, we construct a benchmark for diagnosing interaction behavior and efficiency, together with a Weighted Success Rate metric that penalizes each query by its derived cost. We further propose a zero-shot MLLM navigator that selectively queries at each decision step only when the expected uncertainty reduction justifies the interaction cost.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 59 of its 68 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

59 of 68 references.

References · 59

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  7. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  9. 2018On Evaluation of Embodied Navigation Agentscited by 526
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  11. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  12. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
38 more
  1. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  2. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  3. 2019Vision-and-Dialog Navigationcited by 246
  4. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  5. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  6. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  7. 2021TEACh: Task-driven Embodied Agents that Chatcited by 194
  8. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  9. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  10. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  11. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  12. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  13. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  14. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  15. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  16. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  17. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  18. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 104
  19. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  20. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  21. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  22. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  23. 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 56
  24. 2025MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigationcited by 55
  25. 2019Just Ask:An Interactive Learning Framework for Vision and Language Navigationcited by 53
  26. 2024Vision-and-Language Navigation via Causal Learningcited by 51
  27. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  28. 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
  29. 2023March in Chat: Interactive Prompting for Remote Embodied Referring Expressioncited by 34
  30. 2020RMM: A Recursive Mental Model for Dialog Navigationcited by 32
  31. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  32. 2020The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigationcited by 26
  33. 2025NavBench: Probing Multimodal Large Language Models for Embodied Navigationcited by 24
  34. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  35. 2025BED-LLM: Intelligent Information Gathering with LLMs and Bayesian Experimental Designcited by 21
  36. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  37. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  38. 2025Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environmentscited by 10

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.