RiverVLN: Phase-Grounded Temporal Vision--Language Navigation for Unmanned Surface Vehicles
Not on the map. Why?
Abstract (arXiv)
Vision-language navigation (VLN) has largely been developed for indoor and terrestrial robots, where language can often be treated as a static goal and motion is approximated by discrete or near-instantaneous actions. These assumptions break down for unmanned surface vehicles (USVs): river navigation requires continuous motion under inertia and limited maneuverability, while long-horizon instructions must be executed through sparse and visually ambiguous maritime landmarks. We introduce RiverVLN, to our knowledge the first benchmark designed for long-horizon USV VLN under continuous riverine motion, and PGT-NAV, a phase-grounded temporal navigation framework for USVs. Rather than directly mapping an entire instruction to motion, PGT-NAV converts it into an ordered sequence of visually verifiable semantic phases and maintains the active phase online through grounded visual and motion evidence. This explicit semantic progress state is fused with visual-motion history and phase-specific grounding to predict six local SE(2) pose increments. The resulting trajectory is executed in a predict-execute-re-observe loop, where the vessel executes toward W3, updates phase and grounding, and replans through a map-based safety layer. Experiments show that PGT-NAV substantially reduces recursive position and heading drift relative to GNM-style and ViNT-style baselines and achieves an average success rate of 0.79 in Unity-ROS closed-loop navigation. Unseen bridge-opening trials and real-world USV experiments further demonstrate that the phase-grounded representation transfers from controlled evaluation to physical USV deployment.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
21 of 30 references.
References · 21
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
- 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 248
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2023ViNT: A Foundation Model for Visual Navigationcited by 222
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
- 2022GNM: A General Navigation Model to Drive Any Robotcited by 171
- 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 156
- 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
- 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
- 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
9 more
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
- 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
- 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
- 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
- 2022One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestonescited by 26
- 2026Sparse Video Generation Propels Real-World Beyond-the-View Vision-Language Navigationcited by 10
- 2025CogDDN: A Cognitive Demand-Driven Navigation with Decision Optimization and Dual-Process Thinkingcited by 7
- 2026GN0: Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language Navigationcited by 6
- 2026WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLNcited by 2
Built on by
Not cited yet.