# A Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cues

arXiv: https://arxiv.org/abs/2207.11717 · HTML: https://mlsanity.com/paper/2207.11717/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 24 Jul 2022.

## Builds on (most cited first, 27 of 27)

- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [VisualBERT: A Simple and Performant Baseline for Vision and Language](https://mlsanity.com/paper/1908.03557/) (2019)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training](https://mlsanity.com/paper/2002.10638/) (2020)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Fine-grained Video-Text Retrieval with Hierarchical Graph Reasoning](https://mlsanity.com/paper/2003.00392/) (2020)
- [Improving Vision-and-Language Navigation with Image-Text Pairs from the Web](https://mlsanity.com/paper/2004.14973/) (2020)
- [Airbert: In-domain Pretraining for Vision-and-Language Navigation](https://mlsanity.com/paper/2108.09105/) (2021)
- [Learning to Navigate in Cities Without a Map](https://mlsanity.com/paper/1804.00168/) (2018)
- [The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation](https://mlsanity.com/paper/2104.04167/) (2021)
- [Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion](https://mlsanity.com/paper/2108.04927/) (2021)
- [Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation](https://mlsanity.com/paper/1906.00347/) (2019)
- [Transferable Representation Learning in Vision-and-Language Navigation](https://mlsanity.com/paper/1908.03409/) (2019)
- [Diagnosing Vision-and-Language Navigation: What Really Matters](https://mlsanity.com/paper/2103.16561/) (2021)
- [Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction](https://mlsanity.com/paper/1811.04179/) (2018)
- [Learning To Follow Directions in Street View](https://mlsanity.com/paper/1903.00401/) (2019)
- [Analyzing Generalization of Vision and Language Navigation to Unseen Outdoor Areas](https://mlsanity.com/paper/2203.13838/) (2022)
- [Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigation](https://mlsanity.com/paper/2007.00229/) (2020)
- [Learning to Stop: A Simple yet Effective Approach to Urban Vision-Language Navigation](https://mlsanity.com/paper/2009.13112/) (2020)
- [MLM: A Benchmark Dataset for Multitask Learning with Multiple Languages and Modalities](https://mlsanity.com/paper/2008.06376/) (2020)
- [Learning to attend in a brain-inspired deep neural network](https://mlsanity.com/paper/1811.09699/) (2018)

## Built on by (most cited first, 5 of 5)

- [VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street View](https://mlsanity.com/paper/2307.06082/) (2023)
- [NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigation](https://mlsanity.com/paper/2411.08579/) (2024)
- [VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigation](https://mlsanity.com/paper/2402.03561/) (2024)
- [Loc4Plan: Locating Before Planning for Outdoor Vision and Language Navigation](https://mlsanity.com/paper/2408.05090/) (2024)
- [Towards General Urban Monitoring with Vision-Language Models: A Review, Evaluation, and a Research Agenda](https://mlsanity.com/paper/2510.12400/) (2025)

## Abstract (arXiv)

In a busy city street, a pedestrian surrounded by distractions can pick out a single sign if it is relevant to their route. Artificial agents in outdoor Vision-and-Language Navigation (VLN) are also confronted with detecting supervisory signal on environment features and location in inputs. To boost the prominence of relevant features in transformer-based architectures without costly preprocessing and pretraining, we take inspiration from priority maps - a mechanism described in neuropsychological studies. We implement a novel priority map module and pretrain on auxiliary tasks using low-sample datasets with high-level representations of routes and environment-related references to urban features. A hierarchical process of trajectory planning - with subsequent parameterised visual boost filtering on visual inputs and prediction of corresponding textual spans - addresses the core challenges of cross-modal alignment and feature-level localisation. The priority map module is integrated into a feature-location framework that doubles the task completion rates of standalone transformers and attains state-of-the-art performance on the Touchdown benchmark for VLN. Code and data are referenced in Appendix C.