mlsanityData to 7 Oct 2026

Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigation

arXiv 2503.11006 ↗First posted 14 Mar 2025cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Vision and Language Navigation (VLN) requires an embodied agent to traverse complex environments by following natural language instructions, demanding accurate alignment between visual observations and linguistic guidance. To address these challenges, we propose a fine grained instruction guided graph reasoning framework (FIGR) that enhances both spatial representation and instruction understanding during navigation. Specifically, an observation graph interaction mechanism is introduced to disentangle angular and visual cues while strengthening directed edge representations through geometric embedding, enabling more reliable spatial reasoning within the navigation graph. The key detail guidance module is implemented as Adaptive Open Vocabulary Guidance (AOVG), where a contextual role parser dynamically identifies location, object, spatial relation, and other contextual cues. This design avoids exact string matching and supports previously unseen entities and compositional expressions. For multilingual instructions, a Multilingual Semantic Adapter (MSA) maps language-specific representations into a shared navigation-semantic space. By jointly integrating structured graph reasoning with instruction critical semantic cues, the proposed approach significantly improves the agent ability to follow complex navigation instructions. On the validation-unseen splits, FIGR achieves 67 SPL on R2R and 64.8 sDTW on RxR, exceeding SPENav by 1 percentage point in SPL and PRET by 2.4 points in sDTW, respectively.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 43 of its 67 references that are in the map. First posted on arXiv 14 Mar 2025.

Builds on

43 of 67 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  10. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  11. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  12. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
31 more
  1. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  2. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  5. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  6. 2020Neural Topological SLAM for Visual Navigationcited by 193
  7. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  8. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  9. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  10. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  11. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  12. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 104
  13. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  14. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  15. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  16. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  17. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  18. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  19. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  20. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  21. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  22. 2023Learning Vision-and-Language Navigation from YouTube Videoscited by 47
  23. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  24. 2023Lana: A Language-Capable Navigator for Instruction Following and Generationcited by 39
  25. 2023KERM: Knowledge Enhanced Reasoning for Vision-and-Language Navigationcited by 38
  26. 2021Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigationcited by 20
  27. 2023Meta-Explore: Exploratory Hierarchical Vision-and-Language Navigation Using Scene Object Spectrum Groundingcited by 18
  28. 2024PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigationcited by 6
  29. 2023Masked Path Modeling for Vision-and-Language Navigationcited by 4
  30. 2023Cross-Modal Information-Guided Network using Contrastive Learning for Point Cloud Registrationcited by 4
  31. 2024PointTalk: Audio-Driven Dynamic Lip Point Cloud for 3D Gaussian-based Talking Head Synthesiscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agentscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigation
  2. 2025Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents
  3. 2025EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks
  4. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agents
  5. 2026GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap
  6. 2026VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation
  7. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
  8. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation