mlsanityData to 7 Oct 2026

Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding

arXiv 2010.07954 ↗First posted 15 Oct 2020cs.CV cs.AI cs.CLcited by 401

Not on the map. Why?

Cited by 30 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Room-Across-Room (RxR), a new Vision-and-Language Navigation (VLN) dataset. RxR is multilingual (English, Hindi, and Telugu) and larger (more paths and instructions) than other VLN datasets. It emphasizes the role of language in VLN by addressing known biases in paths and eliciting more references to visible entities. Furthermore, each word in an instruction is time-aligned to the virtual poses of instruction creators and validators. We establish baseline scores for monolingual and multilingual settings and multitask learning when including Room-to-Room annotations. We also provide results for a model that learns from synchronized pose traces by focusing only on portions of the panorama attended to in human demonstrations. The size, scope and detail of RxR dramatically expands the frontier for research on embodied language agents in simulated, photo-realistic environments.

Key facts. Cited by 401 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 35; 2022: 39; 2023: 59; 2024: 64; 2025: 91; 2026: 113.

401 papers.

Sort:

By area:

  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  2. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  3. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  4. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
  5. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  6. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  7. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  8. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  9. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  10. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  11. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  12. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 201
38 more
  1. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  2. 2023LanguageMPC: Large Language Models as Decision Makers for Autonomous Drivingcited by 188
  3. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  4. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  5. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 164
  6. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  7. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 150
  8. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  9. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  10. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  11. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
  12. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 125
  13. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  14. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  15. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  16. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  17. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  18. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  19. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  20. 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
  21. 2023HomeRobot: Open-Vocabulary Mobile Manipulationcited by 104
  22. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  23. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  24. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 92
  25. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  26. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  27. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  28. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  29. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 87
  30. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  31. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  32. 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 80
  33. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  34. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  35. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  36. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 72
  37. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  38. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69

50 most cited of 401 All 401 (JSON)