mlsanityData to 7 Oct 2026

PRET: Planning with Directed Fidelity Trajectory for Vision and Language Navigation

arXiv 2407.11487 ↗First posted 16 Jul 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Vision and language navigation is a task that requires an agent to navigate according to a natural language instruction. Recent methods predict sub-goals on constructed topology map at each step to enable long-term action planning. However, they suffer from high computational cost when attempting to support such high-level predictions with GCN-like models. In this work, we propose an alternative method that facilitates navigation planning by considering the alignment between instructions and directed fidelity trajectories, which refers to a path from the initial node to the candidate locations on a directed graph without detours. This planning strategy leads to an efficient model while achieving strong performance. Specifically, we introduce a directed graph to illustrate the explored area of the environment, emphasizing directionality. Then, we firstly define the trajectory representation as a sequence of directed edge features, which are extracted from the panorama based on the corresponding orientation. Ultimately, we assess and compare the alignment between instruction and different trajectories during navigation to determine the next navigation target. Our method outperforms previous SOTA method BEVBert on RxR dataset and is comparable on R2R dataset while largely reducing the computational cost. Code is available: https://github.com/iSEE-Laboratory/VLN-PRET.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 42 of its 47 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

42 of 47 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  7. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  8. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  11. 2022Efficiently Scaling Transformer Inferencecited by 529
  12. 2018On Evaluation of Embodied Navigation Agentscited by 526
30 more
  1. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  2. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  3. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  4. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  5. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  6. 2020Learning to Explore using Active Neural SLAMcited by 308
  7. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  8. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  9. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  10. 2019Vision-and-Dialog Navigationcited by 246
  11. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  12. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  13. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  14. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  15. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  16. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  17. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  18. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  19. 2019The Regretful Agent: Heuristic-Aided Navigation through Progress Estimationcited by 104
  20. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 102
  21. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  22. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  23. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
  24. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  25. 2021Less is More: Generating Grounded Navigation Instructions from Landmarkscited by 62
  26. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  27. 2023GeoVLN: Learning Geometry-Enhanced Visual Representation with Slot Attention for Vision-and-Language Navigationcited by 24
  28. 2021Multimodal Transformer with Variable-length Memory for Vision-and-Language Navigationcited by 20
  29. 2023Meta-Explore: Exploratory Hierarchical Vision-and-Language Navigation Using Scene Object Spectrum Groundingcited by 18
  30. 2023Masked Path Modeling for Vision-and-Language Navigationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2025Multimodal Fusion and Vision-Language Models: A Survey for Robot Visioncited by 37
  2. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  3. 2025Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigationcited by 3
  4. 2025MM-Nav: Multi-View VLA Model for Robust Visual Navigation via Multi-Expert Learningcited by 3
  5. 2025Fine-Grained Instruction-Guided Graph Reasoning for Vision-and-Language Navigationcited by 1
  6. 2026ReCAPA: Hierarchical Predictive Correction to Mitigate Cascading Failurescited by 1