mlsanityData to 7 Oct 2026

Iterative Vision-and-Language Navigation

arXiv 2210.03087 ↗First posted 6 Oct 2022cs.CV cs.CL cs.ROcited by 34

Not on the map. Why?

Abstract (arXiv)

We present Iterative Vision-and-Language Navigation (IVLN), a paradigm for evaluating language-guided agents navigating in a persistent environment over time. Existing Vision-and-Language Navigation (VLN) benchmarks erase the agent's memory at the beginning of every episode, testing the ability to perform cold-start navigation with no prior information. However, deployed robots occupy the same environment for long periods of time. The IVLN paradigm addresses this disparity by training and evaluating VLN agents that maintain memory across tours of scenes that consist of up to 100 ordered instruction-following Room-to-Room (R2R) episodes, each defined by an individual language instruction and a target path. We present discrete and continuous Iterative Room-to-Room (IR2R) benchmarks comprising about 400 tours each in 80 indoor scenes. We find that extending the implicit memory of high-performing transformer VLN agents is not sufficient for IVLN, but agents that build maps can benefit from environment persistence, motivating a renewed focus on map-building agents in VLN.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 41 of its 61 references that are in the map. First posted on arXiv 6 Oct 2022.

Builds on

41 of 61 references.

References · 41

  1. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  2. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  3. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  5. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  7. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  8. 2018On Evaluation of Embodied Navigation Agentscited by 526
  9. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  10. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  11. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  12. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
29 more
  1. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  2. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  3. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  4. 2019Vision-and-Dialog Navigationcited by 246
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  6. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  7. 2019Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mappingcited by 181
  8. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  9. 2020Improving Vision-and-Language Navigation with Image-Text Pairs from the Webcited by 177
  10. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  11. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  12. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  13. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  14. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  15. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  16. 2019Embodied Question Answering in Photorealistic Environments with Point Cloud Perceptioncited by 103
  17. 2019General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warpingcited by 102
  18. 2021Visual Room Rearrangementcited by 101
  19. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  20. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  21. 2020Sim-to-Real Transfer for Vision-and-Language Navigationcited by 88
  22. 2018RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentationcited by 65
  23. 2020Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Viewscited by 57
  24. 2020BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Stepscited by 49
  25. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  26. 2021Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigationcited by 47
  27. 2018Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Predictioncited by 35
  28. 2020The RobotSlang Benchmark: Dialog-guided Robot Localization and Navigationcited by 26
  29. 2021A Comparison of Modern General-Purpose Visual SLAM Approachescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 13; 2025: 6; 2026: 10.

34 papers.

Sort:

By area:

  1. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  2. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  3. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  4. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  5. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
  6. 2025OctoNav: Towards Generalist Embodied Navigationcited by 33
  7. 2024OVER-NAV: Elevating Iterative Vision-and-Language Navigation with Open-Vocabulary Detection and StructurEd Representationcited by 17
  8. 2025VLNVerse: A Benchmark for Vision-Language Navigation with Versatile, Embodied, Realistic Simulation and Evaluationcited by 17
  9. 2025Towards Physically Executable 3D Gaussian for Embodied Navigationcited by 16
  10. 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
  11. 2023RREx-BoT: Remote Referring Expressions with a Bag of Trickscited by 12
  12. 2024Malicious Path Manipulations via Exploitation of Representation Vulnerabilities of Vision-Language Navigation Systemscited by 10
22 more
  1. 2023ESceme: Vision-and-Language Navigation with Episodic Scene Memorycited by 9
  2. 2024Why Only Text: Empowering Vision-and-Language Navigation with Multi-modal Promptscited by 9
  3. 2024Tell Me Where You Are: Multimodal LLMs Meet Place Recognitioncited by 7
  4. 2025NavQ: Learning a Q-Model for Foresighted Vision-and-Language Navigationcited by 6
  5. 2022Learning-Augmented Model-Based Planning for Visual Explorationcited by 5
  6. 2025Hierarchical Instruction-aware Embodied Visual Trackingcited by 4
  7. 2025Dream to Recall: Imagination-Guided Experience Retrieval for Memory-Persistent Vision-and-Language Navigationcited by 3
  8. 2022Comparison of Model-Free and Model-Based Learning-Informed Planning for PointGoal Navigationcited by 2
  9. 2024Continual Vision-and-Language Navigationcited by 2
  10. 2024NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired Peoplecited by 2
  11. 2026FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigationcited by 2
  12. 2024IVLMap: Instance-Aware Visual Language Grounding for Consumer Robot Navigationcited by 1
  13. 2024The Wallpaper is Ugly: Indoor Localization using Vision and Languagecited by 1
  14. 2026All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptationcited by 1
  15. 2026Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigationcited by 1
  16. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  17. 2026Towards Open Environments and Instructions: General Vision-Language Navigation via Fast-Slow Interactive Reasoningcited by 0
  18. 2026VLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operatorcited by 0
  19. 2026pFedNavi: Structure-Aware Personalized Federated Vision-Language Navigation for Embodied AIcited by 0
  20. 2026AISPO: Enhancing Depth Reliability for Robotic Manipulation of Non-Lambertian Objects via Affine-Invariant Shape Priorcited by 0
  21. 2026DART-VLN: Test-Time Memory Decay and Anti-Loop Regularization for Discrete Vision-Language Navigationcited by 0
  22. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Methodcited by 0