mlsanityData to 7 Oct 2026

Towards Navigation by Reasoning over Spatial Configurations

arXiv 2105.06839 ↗First posted 14 May 2021cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

We deal with the navigation problem where the agent follows natural language instructions while observing the environment. Focusing on language understanding, we show the importance of spatial semantics in grounding navigation instructions into visual perceptions. We propose a neural agent that uses the elements of spatial configurations and investigate their influence on the navigation agent's reasoning ability. Moreover, we model the sequential execution order and align visual objects with spatial configurations in the instruction. Our neural agent improves strong baselines on the seen environments and shows competitive performance on the unseen environments. Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 14 May 2021.

Builds on

18 of 25 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  6. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 304
  7. 2019Vision-and-Dialog Navigationcited by 246
  8. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  9. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  10. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 173
  11. 2020Sub-Instruction Aware Vision-and-Language Navigationcited by 64
  12. 2019Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigationcited by 54
6 more
  1. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  2. 2020BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Stepscited by 49
  3. 2020Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Followingcited by 18
  4. 2020SRLGRN: Semantic Role Labeling Graph Reasoning Networkcited by 12
  5. 2018Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agentscited by 5
  6. 2020From Spatial Relations to Spatial Configurationscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 5; 2025: 2.

14 papers.

Sort:

By area:

  1. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  2. 2023MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Predictioncited by 49
  3. 2022Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoningcited by 32
  4. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 26
  5. 2023VLN-Trans: Translator for the Vision and Language Navigation Agentcited by 25
  6. 2024NavHint: Vision and Language Navigation Agent with a Hint Generatorcited by 12
  7. 2023MLANet: Multi-Level Attention Network with Sub-instruction for Continuous Vision-and-Language Navigationcited by 11
  8. 2022LOViS: Learning Orientation and Visual Signals for Vision and Language Navigationcited by 7
  9. 2024Narrowing the Gap between Vision and Action in Navigationcited by 5
  10. 2025NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularizationcited by 5
  11. 2023DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Textcited by 3
  12. 2025Vision-and-Language Navigation with Analogical Textual Descriptions in LLMscited by 3
2 more
  1. 2023Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extractioncited by 2
  2. 2024NAVCON: A Cognitively Inspired and Linguistically Grounded Corpus for Vision and Language Navigationcited by 0