mlsanityData to 7 Oct 2026

LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation

arXiv 2209.12723 ↗First posted 26 Sep 2022cs.CV cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Understanding spatial and visual information is essential for a navigation agent who follows natural language instructions. The current Transformer-based VLN agents entangle the orientation and vision information, which limits the gain from the learning of each information source. In this paper, we design a neural agent with explicit Orientation and Vision modules. Those modules learn to ground spatial information and landmark mentions in the instructions to the visual environment more effectively. To strengthen the spatial reasoning and visual perception of the agent, we design specific pre-training tasks to feed and better utilize the corresponding modules in our final navigation model. We evaluate our approach on both Room2room (R2R) and Room4room (R4R) datasets and achieve the state of the art results on both benchmarks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 27 of its 30 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

27 of 30 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  9. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  10. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 300
  11. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  12. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
15 more
  1. 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 228
  2. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  3. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  4. 2019Stay on the Path: Instruction Fidelity in Vision-and-Language Navigationcited by 129
  5. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 129
  6. 2020Language and Visual Entity Relationship Graph for Agent Navigationcited by 100
  7. 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
  8. 2020Object-and-Action Aware Model for Visual Language Navigationcited by 77
  9. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  10. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 69
  11. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 64
  12. 2021Neighbor-view Enhanced Model for Vision and Language Navigationcited by 50
  13. 2021Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Informationcited by 31
  14. 2021Towards Navigation by Reasoning over Spatial Configurationscited by 14
  15. 2020From Spatial Relations to Spatial Configurationscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 5; 2025: 1.

7 papers.

  1. 2024Common Sense Reasoning for Deepfake Detectioncited by 64
  2. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 26
  3. 2023VLN-Trans: Translator for the Vision and Language Navigation Agentcited by 25
  4. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  5. 2025FlexVLN: Flexible Adaptation for Diverse Vision-and-Language Navigation Taskscited by 16
  6. 2024DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learningcited by 7
  7. 2024Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents
  2. 2026AgentVLN: Towards Agentic Vision-and-Language Navigation
  3. 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agents
  4. 2025A Navigation Framework Utilizing Vision-Language Models
  5. 2025Do Visual Imaginations Improve Vision-and-Language Navigation Agents?
  6. 2026AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
  7. 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Models
  8. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning