mlsanityData to 14 Oct 2026

One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints

arXiv 2609.06476 ↗First posted 6 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate unseen environments by following natural language instructions. Current zero-shot VLN-CE methods either rely on pre-trained waypoint predictors or require multiple queries to large models per step. To address prohibitive inference latency and computational overhead, we propose O2C-Nav, an efficient zero-shot navigation framework that calls only a single large model once per decision step. Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers. The MLLM selects a waypoint or generates a fallback target bounding box at each step, while a low-level Fast Marching Method (FMM) planner converts the selected target into an executable collision-free path. This paradigm provides the model with concrete spatial perception and explicit memory while significantly reducing the visual processing load. Extensive evaluations on the R2R-CE and RxR-CE benchmarks demonstrate that O2C-Nav outperforms current state-of-the-art zero-shot methods, highlighting its great potential for real-time robotic deployment. Code is available at https://github.com/kkpsq/O2C-Nav-Code.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 37 references that are in the map. First posted on arXiv 6 Sep 2026.

Builds on

33 of 37 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  6. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  7. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  8. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  9. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  10. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
  11. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
  12. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
21 more
  1. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  2. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
  3. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  4. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
  5. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 105
  6. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  7. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  8. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  9. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 62
  10. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 62
  11. 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
  12. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
  13. 2021Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigationcited by 47
  14. 2025Nav-R1: Reasoning and Navigation in Embodied Scenescited by 38
  15. 2025OmniNav: A Unified Framework for Prospective Exploration and Visual-Language Navigationcited by 36
  16. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  17. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
  18. 2025LaViRA: Language-Vision-Robot Actions Translation for Zero-Shot Vision Language Navigation in Continuous Environmentscited by 9
  19. 2025STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimizationcited by 6
  20. 2026Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigationcited by 4
  21. 2025UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Modelcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Boosting Zero-Shot VLN via Abstract Obstacle Map-Based Waypoint Prediction with TopoGraph-and-VisitInfo-Aware Prompting
  2. 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness
  3. 2026LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
  4. 2026C2Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation
  5. 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation
  6. 2026Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation
  7. 2026ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models
  8. 2026SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment