mlsanityData to 14 Oct 2026

Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Prediction

arXiv 1811.04179 ↗First posted 10 Nov 2018cs.RO cs.AI cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

We propose an approach for mapping natural language instructions and raw observations to continuous control of a quadcopter drone. Our model predicts interpretable position-visitation distributions indicating where the agent should go during execution and where it should stop, and uses the predicted distributions to select the actions to execute. This two-step model decomposition allows for simple and efficient training using a combination of supervised learning and imitation learning. We evaluate our approach with a realistic drone simulator, and demonstrate absolute task-completion accuracy improvements of 16.85% over two state-of-the-art instruction-following methods.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 11; 2022: 9; 2023: 3; 2024: 6; 2025: 3; 2026: 2.

35 papers.

Sort:

By area:

  1. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  2. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  3. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 138
  4. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  5. 2021FILM: Following Instructions in Language with Modular Methodscited by 121
  6. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  7. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 119
  8. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  9. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  10. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  11. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  12. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
23 more
  1. 2021Visual Navigation with Spatial Attentioncited by 43
  2. 2022Iterative Vision-and-Language Navigationcited by 35
  3. 2021Vision-Language Navigation: A Survey and Taxonomycited by 34
  4. 2022Episodic Memory Question Answeringcited by 32
  5. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  6. 2022Continual Learning for Instruction Following from Realtime Feedbackcited by 22
  7. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 20
  8. 2024Human-Aware Vision-and-Language Navigation: Bridging Simulation to Reality with Dynamic Human Interactionscited by 19
  9. 2024NavAgent: Multi-scale Urban Street View Fusion For UAV Embodied Vision-and-Language Navigationcited by 18
  10. 2022Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Taskscited by 17
  11. 2024CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissectioncited by 15
  12. 2021Trajectory Prediction with Linguistic Representationscited by 13
  13. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  14. 2025Memory-Centric Embodied Question Answeringcited by 12
  15. 2022Don't Copy the Teacher: Data and Model Challenges in Embodied Dialoguecited by 10
  16. 2023Multitask Multimodal Prompted Training for Interactive Embodied Task Completioncited by 9
  17. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  18. 2025HA-VLN 2.0: An Open Benchmark and Leaderboard for Human-Aware Navigation in Discrete and Continuous Environments with Dynamic Multi-Human Interactionscited by 7
  19. 2022A Priority Map for Vision-and-Language Navigation with Trajectory Plans and Feature-Location Cuescited by 5
  20. 2022Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledgecited by 3
  21. 2026Memory-Guided View Refinement for Dynamic Human-in-the-loop EQAcited by 1
  22. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  23. 2023Core Challenges in Embodied Vision-Language Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learning
  2. 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning
  3. 2025SINGER: An Onboard Generalist Vision-Language Navigation Policy for Drones
  4. 2026Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents
  5. 2026Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations
  6. 2024Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features
  7. 2019Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations
  8. 2020Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator