# Loc4Plan: Locating Before Planning for Outdoor Vision and Language Navigation

arXiv: https://arxiv.org/abs/2408.05090 · HTML: https://mlsanity.com/paper/2408.05090/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models](https://mlsanity.com/paper/2305.16986/) (2023)
- [History Aware Multimodal Transformer for Vision-and-Language Navigation](https://mlsanity.com/paper/2110.13309/) (2021)
- [Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout](https://mlsanity.com/paper/1904.04195/) (2019)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training](https://mlsanity.com/paper/2002.10638/) (2020)
- [Improving Vision-and-Language Navigation with Image-Text Pairs from the Web](https://mlsanity.com/paper/2004.14973/) (2020)
- [Learning to Navigate in Cities Without a Map](https://mlsanity.com/paper/1804.00168/) (2018)
- [Gated-Attention Architectures for Task-Oriented Language Grounding](https://mlsanity.com/paper/1706.07230/) (2017)
- [Bird's-Eye-View Scene Graph for Vision-Language Navigation](https://mlsanity.com/paper/2308.04758/) (2023)
- [Sub-Instruction Aware Vision-and-Language Navigation](https://mlsanity.com/paper/2004.02707/) (2020)
- [Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion](https://mlsanity.com/paper/2108.04927/) (2021)
- [Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation](https://mlsanity.com/paper/1906.00347/) (2019)
- [BabyWalk: Going Farther in Vision-and-Language Navigation by Taking Baby Steps](https://mlsanity.com/paper/2005.04625/) (2020)
- [Lana: A Language-Capable Navigator for Instruction Following and Generation](https://mlsanity.com/paper/2303.08409/) (2023)
- [Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas](https://mlsanity.com/paper/2203.13838/) (2022)
- [Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation](https://mlsanity.com/paper/2007.00229/) (2020)
- [Generating Landmark Navigation Instructions from Maps as a Graph-to-Text Problem](https://mlsanity.com/paper/2012.15329/) (2020)
- [VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigation](https://mlsanity.com/paper/2402.03561/) (2024)
- [A Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cues](https://mlsanity.com/paper/2207.11717/) (2022)

## Built on by (most cited first, 11 of 11)

- [Where am I? Cross-View Geo-localization with Natural Language Descriptions](https://mlsanity.com/paper/2412.17007/) (2024)
- [Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System](https://mlsanity.com/paper/2506.19433/) (2025)
- [Distilling LLM Prior to Flow Model for Generalizable Agent's Imagination in Object Goal Navigation](https://mlsanity.com/paper/2508.09423/) (2025)
- [City Navigation in the Wild: Exploring Emergent Navigation from Web-Scale Knowledge in MLLMs](https://mlsanity.com/paper/2512.15933/) (2025)
- [Global Cross-Modal Geo-Localization: A Million-Scale Dataset and a Physical Consistency Learning Framework](https://mlsanity.com/paper/2603.08491/) (2026)
- [A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigation](https://mlsanity.com/paper/2607.09792/) (2026)
- [DialogueVPR: Towards Conversational Visual Place Recognition](https://mlsanity.com/paper/2607.14115/) (2026)
- [Rule-VLN: Bridging Perception and Compliance via Semantic Reasoning and Geometric Rectification](https://mlsanity.com/paper/2604.16993/) (2026)
- [MAPS: Multi-Anchor Projection Similarity for Joint Vision-Language Geo-Localization](https://mlsanity.com/paper/2606.22543/) (2026)
- [Trajectory-aware Cross-view Geo-localization with Sequential Observations](https://mlsanity.com/paper/2607.15491/) (2026)
- [NavVerse: Benchmarking Indoor-to-Outdoor Embodied Navigation in Continuous Robot Simulation](https://mlsanity.com/paper/2607.19695/) (2026)

## Abstract (arXiv)

Vision and Language Navigation (VLN) is a challenging task that requires agents to understand instructions and navigate to the destination in a visual environment.One of the key challenges in outdoor VLN is keeping track of which part of the instruction was completed. To alleviate this problem, previous works mainly focus on grounding the natural language to the visual input, but neglecting the crucial role of the agent's spatial position information in the grounding process. In this work, we first explore the substantial effect of spatial position locating on the grounding of outdoor VLN, drawing inspiration from human navigation. In real-world navigation scenarios, before planning a path to the destination, humans typically need to figure out their current location. This observation underscores the pivotal role of spatial localization in the navigation process. In this work, we introduce a novel framework, Locating be for Planning (Loc4Plan), designed to incorporate spatial perception for action planning in outdoor VLN tasks. The main idea behind Loc4Plan is to perform the spatial localization before planning a decision action based on corresponding guidance, which comprises a block-aware spatial locating (BAL) module and a spatial-aware action planning (SAP) module. Specifically, to help the agent perceive its spatial location in the environment, we propose to learn a position predictor that measures how far the agent is from the next intersection for reflecting its position, which is achieved by the BAL module. After the locating process, we propose the SAP module to incorporate spatial information to ground the corresponding guidance and enhance the precision of action planning. Extensive experiments on the Touchdown and map2seq datasets show that the proposed Loc4Plan outperforms the SOTA methods.