mlsanityData to 7 Oct 2026

ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments

arXiv 2304.03047 ↗First posted 6 Apr 2023cs.CV cs.CL cs.ROcited by 165

Not on the map. Why?

Abstract (arXiv)

Vision-language navigation is a task that requires an agent to follow instructions to navigate in environments. It becomes increasingly crucial in the field of embodied AI, with potential applications in autonomous navigation, search and rescue, and human-robot interaction. In this paper, we propose to address a more practical yet challenging counterpart setting - vision-language navigation in continuous environments (VLN-CE). To develop a robust VLN-CE agent, we propose a new navigation framework, ETPNav, which focuses on two critical skills: 1) the capability to abstract environments and generate long-range navigation plans, and 2) the ability of obstacle-avoiding control in continuous environments. ETPNav performs online topological mapping of environments by self-organizing predicted waypoints along a traversed path, without prior environmental experience. It privileges the agent to break down the navigation procedure into high-level planning and low-level control. Concurrently, ETPNav utilizes a transformer-based cross-modal planner to generate navigation plans based on topological maps and instructions. The plan is then performed through an obstacle-avoiding controller that leverages a trial-and-error heuristic to prevent navigation from getting stuck in obstacles. Experimental results demonstrate the effectiveness of the proposed method. ETPNav yields more than 10% and 20% improvements over prior state-of-the-art on R2R-CE and RxR-CE datasets, respectively. Our code is available at https://github.com/MarSaKi/ETPNav.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). Builds on 77 of its 100 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

77 of 100 references.

References · 77

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  9. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  10. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  12. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
38 more
  1. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  2. 2018On Evaluation of Embodied Navigation Agentscited by 526
  3. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  4. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  5. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  6. 2022Multimodal Learning with Transformers: A Surveycited by 399
  7. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  8. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  9. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  10. 2020Learning to Explore using Active Neural SLAMcited by 308
  11. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  12. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  13. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  14. 2019Vision-and-Dialog Navigationcited by 246
  15. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  16. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  17. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  18. 2020Neural Topological SLAM for Visual Navigationcited by 193
  19. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  20. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  21. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  22. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  23. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  24. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  25. 2021FILM: Following Instructions in Language with Modular Methodscited by 119
  26. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  27. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  28. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  29. 2022Navigating to Objects in the Real Worldcited by 104
  30. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 102
  31. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  32. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  33. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  34. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  35. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  36. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  37. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  38. 2018Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigationcited by 84

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 35; 2025: 47; 2026: 79.

165 papers.

Sort:

By area:

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  2. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
  3. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  4. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  5. 2024Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Taskscited by 146
  6. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  7. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  8. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  9. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  10. 2023Learning to Model the World with Languagecited by 66
  11. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  12. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
38 more
  1. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  2. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
  3. 2024Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMscited by 60
  4. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  5. 2024GaussNav: Gaussian Splatting for Visual Navigationcited by 49
  6. 2024Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environmentscited by 49
  7. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 43
  8. 2025NavMorph: A Self-Evolving World Model for Vision-and-Language Navigation in Continuous Environmentscited by 41
  9. 2024Sim-to-Real Transfer via 3D Feature Fields for Vision-and-Language Navigationcited by 34
  10. 2025SmartWay: Enhanced Waypoint Prediction and Backtracking for Zero-Shot Vision-and-Language Navigationcited by 29
  11. 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 26
  12. 2025Rethinking the Embodied Gap in Vision-and-Language Navigation: A Holistic Study of Physical and Visual Disparitiescited by 24
  13. 2025Embodied AI: From LLMs to World Modelscited by 23
  14. 2026ABot-N0: Technical Report on the VLA Foundation Model for Versatile Embodied Navigationcited by 23
  15. 2025Dynam3D: Dynamic Layered 3D Tokens Empower VLM for Vision-and-Language Navigationcited by 21
  16. 2024Controllable Navigation Instruction Generation with Chain of Thought Promptingcited by 20
  17. 2024Navigation Instruction Generation with BEV Perception and Large Language Modelscited by 20
  18. 2025Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 20
  19. 2024Vision-Language Navigation with Energy-Based Policycited by 19
  20. 2024NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigationcited by 18
  21. 2024g3D-LF: Generalizable 3D-Language Feature Fields for Embodied Taskscited by 18
  22. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  23. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  24. 2025AstraNav-World: World Model for Foresight Control and Consistencycited by 17
  25. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  26. 2025MobileVLA-R1: Reinforcing Vision-Language-Action for Mobile Robotscited by 15
  27. 2025NavForesee: A Unified Vision-Language World Model for Hierarchical Planning and Dual-Horizon Navigation Predictioncited by 15
  28. 2023Advances in Embodied Navigation Using Large Language Models: A Surveycited by 14
  29. 2025DOPE: Dual Object Perception-Enhancement Network for Vision-and-Language Navigationcited by 13
  30. 2025Advancing Audio-Visual Navigation Through Multi-Agent Collaboration in 3D Environmentscited by 13
  31. 2024NavHint: Vision and Language Navigation Agent with a Hint Generatorcited by 12
  32. 2024Part2Object: Hierarchical Unsupervised 3D Instance Segmentationcited by 12
  33. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  34. 2024Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigationcited by 11
  35. 2025ActiveVLN: Towards Active Exploration via Multi-Turn RL in Vision-and-Language Navigationcited by 11
  36. 2024Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environmentscited by 10
  37. 2025Ground-level Viewpoint Vision-and-Language Navigation in Continuous Environmentscited by 10
  38. 2025GC-VLN: Instruction as Graph Constraints for Training-free Vision-and-Language Navigationcited by 10

50 most cited of 165 All 165 (JSON)