mlsanityData to 7 Oct 2026

General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping

arXiv 1907.05446 ↗First posted 11 Jul 2019cs.RO cs.AI cs.CLcited by 102

Not on the map. Why?

Abstract (arXiv)

In instruction conditioned navigation, agents interpret natural language and their surroundings to navigate through an environment. Datasets for studying this task typically contain pairs of these instructions and reference trajectories. Yet, most evaluation metrics used thus far fail to properly account for the latter, relying instead on insufficient similarity comparisons. We address fundamental flaws in previously used metrics and show how Dynamic Time Warping (DTW), a long known method of measuring similarity between two time series, can be used for evaluation of navigation agents. For such, we define the normalized Dynamic Time Warping (nDTW) metric, that softly penalizes deviations from the reference path, is naturally sensitive to the order of the nodes composing each path, is suited for both continuous and graph-based evaluations, and can be efficiently calculated. Further, we define SDTW, which constrains nDTW to only successful paths. We collect human similarity judgments for simulated paths and find nDTW correlates better with human rankings than all other metrics. We also demonstrate that using nDTW as a reward signal for Reinforcement Learning navigation agents improves their performance on both the Room-to-Room (R2R) and Room-for-Room (R4R) datasets. The R4R results in particular highlight the superiority of SDTW over previous success-constrained metrics.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 18; 2023: 11; 2024: 17; 2025: 19; 2026: 23.

102 papers.

Sort:

By area:

  1. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  2. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  3. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  4. 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
  5. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  6. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  7. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  8. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  9. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  10. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  11. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  12. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
38 more
  1. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  2. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  3. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  4. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  5. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  6. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  7. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  8. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  9. 2022Retrospectives on the Embodied AI Workshopcited by 46
  10. 2023Improving Vision-and-Language Navigation by Generating Future-View Image Semanticscited by 46
  11. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  12. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  13. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  14. 2022Reinforced Structured State-Evolution for Vision-Language Navigationcited by 36
  15. 2022Iterative Vision-and-Language Navigationcited by 34
  16. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
  17. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  18. 20221st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)cited by 30
  19. 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learningcited by 30
  20. 2021Interactive Learning from Activity Descriptioncited by 26
  21. 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 26
  22. 2023VLN-Trans: Translator for the Vision and Language Navigation Agentcited by 25
  23. 2023Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigationcited by 25
  24. 2022FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigationcited by 22
  25. 2022Ground then Navigate: Language-guided Navigation in Dynamic Scenescited by 20
  26. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  27. 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
  28. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  29. 2021Contrastive Instruction-Trajectory Learning for Vision-Language Navigationcited by 16
  30. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  31. 2025A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AIcited by 16
  32. 2025Do Visual Imaginations Improve Vision-and-Language Navigation Agents?cited by 14
  33. 2021VISITRON: Visual Semantics-Aligned Interactively Trained Object-Navigatorcited by 13
  34. 2024NavHint: Vision and Language Navigation Agent with a Hint Generatorcited by 12
  35. 2025ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigationcited by 11
  36. 2025MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluationcited by 11
  37. 2022CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representationscited by 10
  38. 2024Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environmentscited by 10

50 most cited of 102 All 102 (JSON)