mlsanityData to 7 Oct 2026

Waypoint Models for Instruction-guided Navigation in Continuous Environments

arXiv 2110.02207 ↗First posted 5 Oct 2021cs.CV cs.CL cs.ROcited by 125

Not on the map. Why?

Abstract (arXiv)

Little inquiry has explicitly addressed the role of action spaces in language-guided visual navigation -- either in terms of its effect on navigation success or the efficiency with which a robotic agent could execute the resulting trajectory. Building on the recently released VLN-CE setting for instruction following in continuous environments, we develop a class of language-conditioned waypoint prediction networks to examine this question. We vary the expressivity of these models to explore a spectrum between low-level actions and continuous waypoint prediction. We measure task performance and estimated execution time on a profiled LoCoBot robot. We find more expressive models result in simpler, faster to execute trajectories, but lower-level actions can achieve better navigation metrics by approximating shortest paths better. Further, our models outperform prior work in VLN-CE and set a new state-of-the-art on the public leaderboard -- increasing success rate by 4% with our best model on this challenging task.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). Builds on 34 of its 40 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

34 of 40 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  8. 2018On Evaluation of Embodied Navigation Agentscited by 526
  9. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  10. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  11. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  12. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
22 more
  1. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  2. 2020Learning to Explore using Active Neural SLAMcited by 308
  3. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  4. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  5. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  6. 2020Neural Topological SLAM for Visual Navigationcited by 193
  7. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  8. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  9. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  10. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  11. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  12. 2018Neural Modular Control for Embodied Question Answeringcited by 56
  13. 2019Combining Optimal Control and Learning for Visual Navigation in Novel Environmentscited by 50
  14. 2020ReLMoGen: Leveraging Motion Generation in Reinforcement Learning for Mobile Manipulationcited by 47
  15. 2021Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigationcited by 47
  16. 2019Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flightcited by 43
  17. 2019PyRobot: An Open-source Robotics Framework for Research and Benchmarkingcited by 39
  18. 2018Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Predictioncited by 35
  19. 2019Learning To Follow Directions in Street Viewcited by 29
  20. 2018Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learningcited by 27
  21. 2019Sim2Real Predictivity: Does Evaluation in Simulation Predict Real-World Performance?cited by 25
  22. 2021Success Weighted by Completion Time: A Dynamics-Aware Evaluation Criteria for Embodied Navigationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 20; 2023: 16; 2024: 18; 2025: 29; 2026: 42.

125 papers.

Sort:

By area:

  1. 2022Visual Language Maps for Robot Navigationcited by 362
  2. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  3. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  4. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  5. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  6. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  7. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  8. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  9. 2022Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scalecited by 116
  10. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  11. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  12. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
38 more
  1. 2022Navigating to Objects in the Real Worldcited by 104
  2. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  3. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  4. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  5. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  6. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  7. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  8. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  9. 2023Learning to Model the World with Languagecited by 66
  10. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  11. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuningcited by 61
  12. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
  13. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  14. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  15. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  16. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 49
  17. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  18. 2022Retrospectives on the Embodied AI Workshopcited by 46
  19. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 43
  20. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  21. 2023Learning Navigational Visual Representations with Semantic Map Supervisioncited by 40
  22. 2022Iterative Vision-and-Language Navigationcited by 34
  23. 20221st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)cited by 30
  24. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  25. 2025CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memorycited by 30
  26. 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 26
  27. 2023Fast-Slow Test-Time Adaptation for Online Vision-and-Language Navigationcited by 25
  28. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
  29. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  30. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  31. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  32. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 16
  33. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  34. 2023Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulationcited by 12
  35. 2025COSMO: Combination of Selective Memorization for Low-cost Vision-and-Language Navigationcited by 12
  36. 2023MLANet: Multi-Level Attention Network with Sub-instruction for Continuous Vision-and-Language Navigationcited by 11
  37. 2023Vision and Language Navigation in the Real World via Online Visual Language Mappingcited by 11
  38. 2023CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigationcited by 10

50 most cited of 125 All 125 (JSON)