mlsanityData to 7 Oct 2026

A Recurrent Vision-and-Language BERT for Navigation

arXiv 2011.13922 ↗First posted 26 Nov 2020cs.CVcited by 300

Not on the map. Why?

Cited by 22 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Accuracy of many visiolinguistic tasks has benefited significantly from the application of vision-and-language(V&L) BERT. However, its application for the task of vision-and-language navigation (VLN) remains limited. One reason for this is the difficulty adapting the BERT architecture to the partially observable Markov decision process present in VLN, requiring history-dependent attention and decision making. In this paper we propose a recurrent BERT model that is time-aware for use in VLN. Specifically, we equip the BERT model with a recurrent function that maintains cross-modal state information for the agent. Through extensive experiments on R2R and REVERIE we demonstrate that our model can replace more complex encoder-decoder models to achieve state-of-the-art results. Moreover, our approach can be generalised to other transformer-based architectures, supports pre-training, and is capable of solving navigation and referring expression tasks simultaneously.

Key facts. Cited by 300 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 52; 2023: 55; 2024: 57; 2025: 59; 2026: 54.

300 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  3. 2022Multimodal Learning with Transformers: A Surveycited by 399
  4. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  5. 2022SQA3D: Situated Question Answering in 3D Scenescited by 303
  6. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  7. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 274
  8. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  9. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  10. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  11. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
  12. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
38 more
  1. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  2. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  3. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  4. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  5. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  6. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  7. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  8. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 117
  9. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 117
  10. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 116
  11. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 115
  12. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  13. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 98
  14. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
  15. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  16. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 89
  17. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  18. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  19. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 85
  20. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  21. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  22. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 80
  23. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  24. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  25. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
  26. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  27. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  28. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  29. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  30. 2022PLOT: Prompt Learning with Optimal Transport for Vision-Language Modelscited by 64
  31. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  32. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 63
  33. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  34. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 60
  35. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  36. 2024Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigationcited by 60
  37. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  38. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 57

50 most cited of 300 All 300 (JSON)