One MLLM, One Call: Efficient Zero-Shot Vision-and-Language Navigation via Spatial-Aware Waypoints
Not on the map. Why?
Abstract (arXiv)
Vision-and-Language Navigation in Continuous Environments (VLN-CE) requires an embodied agent to navigate unseen environments by following natural language instructions. Current zero-shot VLN-CE methods either rely on pre-trained waypoint predictors or require multiple queries to large models per step. To address prohibitive inference latency and computational overhead, we propose O2C-Nav, an efficient zero-shot navigation framework that calls only a single large model once per decision step. Our approach introduces a training-free structured waypoint generator and a novel abstract representation that projects sparse, history-aware candidate waypoints directly onto RGB images as visual markers. The MLLM selects a waypoint or generates a fallback target bounding box at each step, while a low-level Fast Marching Method (FMM) planner converts the selected target into an executable collision-free path. This paradigm provides the model with concrete spatial perception and explicit memory while significantly reducing the visual processing load. Extensive evaluations on the R2R-CE and RxR-CE benchmarks demonstrate that O2C-Nav outperforms current state-of-the-art zero-shot methods, highlighting its great potential for real-time robotic deployment. Code is available at https://github.com/kkpsq/O2C-Nav-Code.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 37 references that are in the map. First posted on arXiv 6 Sep 2026.
Builds on
33 of 37 references.
References · 33
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
- 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
21 more
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
- 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 105
- 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
- 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
- 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
- 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 62
- 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 62
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 62
- 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
- 2021Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigationcited by 47
- 2025Nav-R1: Reasoning and Navigation in Embodied Scenescited by 38
- 2025OmniNav: A Unified Framework for Prospective Exploration and Visual-Language Navigationcited by 36
- 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
- 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10
- 2025LaViRA: Language-Vision-Robot Actions Translation for Zero-Shot Vision Language Navigation in Continuous Environmentscited by 9
- 2025STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimizationcited by 6
- 2026Three-Step Nav: A Hierarchical Global-Local Planner for Zero-Shot Vision-and-Language Navigationcited by 4
- 2025UNeMo: Collaborative Visual-Language Reasoning and Navigation via a Multimodal World Modelcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Boosting Zero-Shot VLN via Abstract Obstacle Map-Based Waypoint Prediction with TopoGraph-and-VisitInfo-Aware Prompting
- 2026AgenticNav: Zero-Shot Vision-and-Language Navigation as a Tool-Calling Harness
- 2026LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs
- 2026C2Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation
- 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigation
- 2026Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation
- 2026ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models
- 2026SFCo-Nav: Efficient Zero-Shot Visual Language Navigation via Collaboration of Slow LLM and Fast Attributed Graph Alignment