mlsanityData to 14 Oct 2026

Landmark-Guided Knowledge for Vision-and-Language Navigation

arXiv 2509.25655 ↗First posted 30 Sep 2025cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision-and-language navigation is one of the core tasks in embodied intelligence, requiring an agent to autonomously navigate in an unfamiliar environment based on natural language instructions. However, existing methods often fail to match instructions with environmental information in complex scenarios, one reason being the lack of common-sense reasoning ability. This paper proposes a vision-and-language navigation method called Landmark-Guided Knowledge (LGK), which introduces an external knowledge base to assist navigation, addressing the misjudgment issues caused by insufficient common sense in traditional methods. Specifically, we first construct a knowledge base containing 630,000 language descriptions and use knowledge Matching to align environmental subviews with the knowledge base, extracting relevant descriptive knowledge. Next, we design a Knowledge-Guided by Landmark (KGL) mechanism, which guides the agent to focus on the most relevant parts of the knowledge by leveraging landmark information in the instructions, thereby reducing the data bias that may arise from incorporating external knowledge. Finally, we propose Knowledge-Guided Dynamic Augmentation (KGDA), which effectively integrates language, knowledge, vision, and historical information. Experimental results demonstrate that the LGK method outperforms existing state-of-the-art methods on the R2R and REVERIE vision-and-language navigation datasets, particularly in terms of navigation error, success rate, and path efficiency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

33 of 45 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  8. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  9. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  10. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  11. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  12. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
21 more
  1. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
  2. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  3. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  4. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  5. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  6. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  7. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  8. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  9. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  10. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 70
  11. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  12. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  13. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  14. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  15. 2022Sound Adversarial Audio-Visual Navigationcited by 48
  16. 2022ADAPT: Vision-Language Navigation with Modality-Aligned Action Promptscited by 45
  17. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 39
  18. 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 27
  19. 2023Measuring Acoustics with Collaborative Multiple Agentscited by 22
  20. 2024Correctable Landmark Discovery via Large Models for Vision-Language Navigationcited by 17
  21. 2024Augmented Commonsense Knowledge for Remote Object Groundingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
  2. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
  3. 2025A Navigation Framework Utilizing Vision-Language Models
  4. 2026SpaceVLN: A Zero-Shot Vision-and-Language Navigation Agent with Online Spatial Cognitive Memory and Reasoning
  5. 2025Aux-Think: Exploring Reasoning Strategies for Data-Efficient Vision-Language Navigation
  6. 2024MiniVLN: Efficient Vision-and-Language Navigation by Progressive Knowledge Distillation
  7. 2025Run, Ruminate, and Regulate: A Dual-process Thinking System for Vision-and-Language Navigation
  8. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation