# Towards Navigation by Reasoning over Spatial Configurations

arXiv: https://arxiv.org/abs/2105.06839 · HTML: https://mlsanity.com/paper/2105.06839/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout](https://mlsanity.com/paper/1904.04195/) (2019)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [Self-Monitoring Navigation Agent via Auxiliary Progress Estimation](https://mlsanity.com/paper/1901.03035/) (2019)
- [Sub-Instruction Aware Vision-and-Language Navigation](https://mlsanity.com/paper/2004.02707/) (2020)
- [Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation](https://mlsanity.com/paper/1906.00347/) (2019)
- [Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequences](https://mlsanity.com/paper/1506.04089/) (2015)
- [BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps](https://mlsanity.com/paper/2005.04625/) (2020)
- [Few-shot Object Grounding and Mapping for Natural Language Robot Instruction Following](https://mlsanity.com/paper/2011.07384/) (2020)
- [SRLGRN: Semantic Role Labeling Graph Reasoning Network](https://mlsanity.com/paper/2010.03604/) (2020)
- [Guided Feature Transformation (GFT): A Neural Language Grounding Module for Embodied Agents](https://mlsanity.com/paper/1805.08329/) (2018)
- [From Spatial Relations to Spatial Configurations](https://mlsanity.com/paper/2007.09557/) (2020)

## Built on by (most cited first, 14 of 14)

- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [MvP: Multi-view Prompting Improves Aspect Sentiment Tuple Prediction](https://mlsanity.com/paper/2305.12627/) (2023)
- [Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning](https://mlsanity.com/paper/2210.16952/) (2022)
- [SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Models](https://mlsanity.com/paper/2410.03878/) (2024)
- [VLN-Trans: Translator for the Vision and Language Navigation Agent](https://mlsanity.com/paper/2302.09230/) (2023)
- [NavHint: Vision and Language Navigation Agent with a Hint Generator](https://mlsanity.com/paper/2402.02559/) (2024)
- [MLANet: Multi-Level Attention Network with Sub-instruction for Continuous Vision-and-Language Navigation](https://mlsanity.com/paper/2303.01396/) (2023)
- [LOViS: Learning Orientation and Visual Signals for Vision and Language Navigation](https://mlsanity.com/paper/2209.12723/) (2022)
- [Narrowing the Gap between Vision and Action in Navigation](https://mlsanity.com/paper/2408.10388/) (2024)
- [NavComposer: Composing Language Instructions for Navigation Trajectories through Action-Scene-Object Modularization](https://mlsanity.com/paper/2507.10894/) (2025)
- [DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text](https://mlsanity.com/paper/2310.12557/) (2023)
- [Vision-and-Language Navigation with Analogical Textual Descriptions in LLMs](https://mlsanity.com/paper/2509.25139/) (2025)
- [Measuring Your ASTE Models in The Wild: A Diversified Multi-domain Dataset For Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2305.17448/) (2023)
- [NAVCON: A Cognitively Inspired and Linguistically Grounded Corpus for Vision and Language Navigation](https://mlsanity.com/paper/2412.13026/) (2024)

## Abstract (arXiv)

We deal with the navigation problem where the agent follows natural language instructions while observing the environment. Focusing on language understanding, we show the importance of spatial semantics in grounding navigation instructions into visual perceptions. We propose a neural agent that uses the elements of spatial configurations and investigate their influence on the navigation agent's reasoning ability. Moreover, we model the sequential execution order and align visual objects with spatial configurations in the instruction. Our neural agent improves strong baselines on the seen environments and shows competitive performance on the unseen environments. Additionally, the experimental results demonstrate that explicit modeling of spatial semantic elements in the instructions can improve the grounding and spatial reasoning of the model.