# RUN through the Streets: A New Dataset and Baseline Models for Realistic Urban Navigation

arXiv: https://arxiv.org/abs/1909.08970 · HTML: https://mlsanity.com/paper/1909.08970/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [Vision-Language Navigation: A Survey and Taxonomy](https://mlsanity.com/paper/2108.11544/) (2021)
- [Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence](https://mlsanity.com/paper/2506.15677/) (2025)
- [Draw Me a Flower: Processing and Grounding Abstraction in Natural Language](https://mlsanity.com/paper/2106.14321/) (2021)
- [DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents](https://mlsanity.com/paper/2210.12511/) (2022)
- [HeGeL: A Novel Dataset for Geo-Location from Hebrew Text](https://mlsanity.com/paper/2307.00509/) (2023)
- [Where Do We Go from Here? Multi-scale Allocentric Relational Inference from Natural Spatial Descriptions](https://mlsanity.com/paper/2402.16364/) (2024)
- [Into the Unknown: Generating Geospatial Descriptions for New Environments](https://mlsanity.com/paper/2406.19967/) (2024)
- [360CityArena: A Realistic Virtual Urban Navigation Benchmark for Embodied Agents](https://mlsanity.com/paper/2608.08814/) (2026)

## Abstract (arXiv)

Following navigation instructions in natural language requires a composition of language, action, and knowledge of the environment. Knowledge of the environment may be provided via visual sensors or as a symbolic world representation referred to as a map. Here we introduce the Realistic Urban Navigation (RUN) task, aimed at interpreting navigation instructions based on a real, dense, urban map. Using Amazon Mechanical Turk, we collected a dataset of 2515 instructions aligned with actual routes over three regions of Manhattan. We propose a strong baseline for the task and empirically investigate which aspects of the neural architecture are important for the RUN success. Our results empirically show that entity abstraction, attention over words and worlds, and a constantly updating world-state, significantly contribute to task accuracy.