mlsanityData to 14 Oct 2026

Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments

arXiv 1811.12354 ↗First posted 29 Nov 2018cs.CV cs.AI cs.CLcited by 239

Not on the map. Why?

Abstract (arXiv)

We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent must first follow navigation instructions in a real-life visual urban environment, and then identify a location described in natural language to find a hidden object at the goal position. The data contains 9,326 examples of English instructions and spatial descriptions paired with demonstrations. Empirical analysis shows the data presents an open challenge to existing methods, and qualitative linguistic analysis shows that the data displays richer use of spatial reasoning compared to related resources.

Key facts. Cited by 239 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 0; 2021: 47; 2022: 48; 2023: 37; 2024: 45; 2025: 37; 2026: 23.

239 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  2. 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
  3. 2021ScanQA: 3D Question Answering for Spatial Scene Understandingcited by 346
  4. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  5. 2022SQA3D: Situated Question Answering in 3D Scenescited by 314
  6. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 271
  7. 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 233
  8. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  9. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 193
  10. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  11. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 169
  12. 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
38 more
  1. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 132
  2. 2021Airbert: In-domain Pretraining for Vision-and-Language Navigationcited by 130
  3. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  4. 2023Vision Language Models in Autonomous Driving: A Survey and Outlookcited by 126
  5. 2022Language Conditioned Spatial Relation Reasoning for 3D Object Groundingcited by 122
  6. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  7. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  8. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  9. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  10. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  11. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  12. 2022Benchmarking Spatial Relationships in Text-to-Image Generationcited by 97
  13. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  14. 2021Language-Aligned Waypoint (LAW) Supervision for Vision-and-Language Navigation in Continuous Environmentscited by 94
  15. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  16. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  17. 2022EnvEdit: Environment Editing for Vision-and-Language Navigationcited by 83
  18. 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 79
  19. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  20. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  21. 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 77
  22. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  23. 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphscited by 73
  24. 2021Vision-Language Navigation with Random Environmental Mixupcited by 70
  25. 2022HOP: History-and-Order Aware Pre-training for Vision-and-Language Navigationcited by 70
  26. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 68
  27. 2023VELMA: Verbalization Embodiment of LLM Agents for Vision and Language Navigation in Street Viewcited by 67
  28. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  29. 2021The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigationcited by 61
  30. 2022A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learningcited by 60
  31. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  32. 2024Toward Visual Grounding: A Surveycited by 52
  33. 2022Learning from Unlabeled 3D Environments for Vision-and-Language Navigationcited by 50
  34. 2021Scene-Intuitive Agent for Remote Embodied Visual Groundingcited by 49
  35. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  36. 2022Aerial Vision-and-Dialog Navigationcited by 49
  37. 2024TopViewRS: Vision-Language Models as Top-View Spatial Reasonerscited by 49
  38. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48

50 most cited of 239 All 239 (JSON)