mlsanityData to 14 Oct 2026

REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments

arXiv 1904.10151 ↗First posted 23 Apr 2019cs.CV cs.CLcited by 333

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

One of the long-term challenges of robotics is to enable robots to interact with humans in the visual world via natural language, as humans are visual animals that communicate through language. Overcoming this challenge requires the ability to perform a wide variety of complex tasks in response to multifarious instructions from humans. In the hope that it might drive progress towards more flexible and powerful human interactions with robots, we propose a dataset of varied and complex robot tasks, described in natural language, in terms of objects visible in a large set of real images. Given an instruction, success requires navigating through a previously-unseen environment to identify an object. This represents a practical challenge, but one that closely reflects one of the core visual problems in robotics. Several state-of-the-art vision-and-language navigation, and referring-expression models are tested to verify the difficulty of this new task, but none of them show promising results because there are many fundamental differences between our task and previous ones. A novel Interactive Navigator-Pointer model is also proposed that provides a strong baseline on the task. The proposed model especially achieves the best performance on the unseen test split, but still leaves substantial room for improvement compared to the human performance.

Key facts. Cited by 333 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 28; 2022: 30; 2023: 59; 2024: 69; 2025: 76; 2026: 70.

333 papers.

Sort:

By area:

  1. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  2. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  3. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Modelscited by 280
  4. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  5. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  6. 2024NaVILA: Legged Robot Vision-Language-Action Model for Navigationcited by 211
  7. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  8. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  9. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  10. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  11. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
  12. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
38 more
  1. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  2. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  3. 2023GridMM: Grid Memory Map for Vision-and-Language Navigationcited by 124
  4. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  5. 2022BEVBert: Multimodal Map Pre-training for Language-guided Navigationcited by 122
  6. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  7. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  8. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  9. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  10. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  11. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  12. 2022EDA: Explicit Text-Decoupling and Dense Alignment for 3D Visual Groundingcited by 99
  13. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 98
  14. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  15. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 94
  16. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 89
  17. 2024MapGPT: Map-Guided Prompting with Adaptive Path Planning for Vision-and-Language Navigationcited by 86
  18. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  19. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  20. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  21. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  22. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  23. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  24. 2024HM3D-OVON: A Dataset and Benchmark for Open-Vocabulary Object Goal Navigationcited by 71
  25. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 70
  26. 2022VLMbench: A Compositional Benchmark for Vision-and-Language Manipulationcited by 68
  27. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  28. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 67
  29. 2024Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigationcited by 65
  30. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  31. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 61
  32. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  33. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
  34. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  35. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 57
  36. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  37. 2022CLIP-Nav: Using CLIP for Zero-Shot Vision-and-Language Navigationcited by 55
  38. 2021SOAT: A Scene- and Object-Aware Transformer for Vision-and-Language Navigationcited by 53

50 most cited of 333 All 333 (JSON)