Robust Vision-Language Navigation with Awareness of Instruction-Execution Progress
Not on the map. Why?
Abstract (arXiv)
In Vision-Language Navigation (VLN) tasks involving multi-step instructions as guidance, an agent must associate its observations with the relevant instruction content at each decision point. Most graph-based methods commonly match candidate trajectories with the entire instruction, allowing already executed content to interfere with subsequent decisions. Although instruction-execution progress provides a direct cue for separating executed and pending content, existing progress estimation approaches often rely on explicit progress supervision, sophisticated progress-learning stages, or costly online reasoning with LLMs. In this paper, we propose a robust VLN framework with Awareness of Instruction-Execution Progress (VLN-AIEP), which couples progress estimation with navigation decision-making. VLN-AIEP associates the visited trajectory with the instruction to suppress embeddings corresponding to executed content, and then matches candidate locations with the pending content. This coupling makes correct navigation decisions depend on accurate progress estimation, allowing the progress estimator to learn directly from navigation decision labels without explicit progress annotations. To support progress estimation, we construct a landmark-anchored trajectory representation that encodes the semantic labels and viewpoint-relative geometry of instruction-referred landmarks. These landmarks serve as spatial references for associating the visited trajectory with executed instruction content. Extensive experiments on R2R, RxR, CVDN, and R2R-CE demonstrate that the proposed method achieves favorable performance across multiple metrics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 56 references that are in the map. First posted on arXiv 9 Oct 2026.
Builds on
51 of 56 references.
References · 51
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
- 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
- 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
- 2019Vision-and-Dialog Navigationcited by 250
38 more
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
- 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
- 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
- 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
- 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
- 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 157
- 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
- 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
- 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 118
- 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 110
- 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 105
- 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
- 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
- 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
- 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
- 2025Embodied Navigation Foundation Modelcited by 72
- 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 70
- 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 66
- 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 64
- 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 62
- 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 61
- 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 55
- 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 50
- 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
- 2023VLN-PETL: Parameter-Efficient Transfer Learning for Vision-and-Language Navigationcited by 23
- 2021Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigationcited by 21
- 2024Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactionscited by 19
- 2024Correctable Landmark Discovery via Large Models for Vision-Language Navigationcited by 17
- 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 17
- 2025Progress-Think: Semantic Progress Reasoning for Vision-Language Navigationcited by 12
- 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigationcited by 10
- 2024PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigationcited by 7
- 2025Fast-SmartWay: Panoramic-Free End-to-End Zero-Shot Vision-and-Language Navigationcited by 7
- 2025Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotationscited by 5
- 2025Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agentscited by 5
- 2026DecoVLN: Decoupling Observation, Reasoning, and Correction for Vision-and-Language Navigationcited by 5
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation
- 2026StageVLN: Spatial and Trajectory Auxiliary Guidance for Efficient Vision-Language Navigation
- 2026Dual-Anchoring: Addressing State Drift in Vision-Language Navigation
- 2026Seek Before You Move: Evidence Seeking for Progress Grounding in Vision-Language Navigation
- 2026What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
- 2026Beyond Waypoints: A Trajectory-Centric Waypointing Paradigm for Vision-Language Navigation
- 2026Learning to Retrieve Navigable Candidates for Efficient Vision-and-Language Navigation
- 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning