mlsanityData to 14 Oct 2026

Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight

arXiv 1910.09664 ↗First posted 21 Oct 2019cs.RO cs.AI cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

We propose a joint simulation and real-world learning framework for mapping navigation instructions and raw first-person observations to continuous control. Our model estimates the need for environment exploration, predicts the likelihood of visiting environment positions during execution, and controls the agent to both explore and visit high-likelihood positions. We introduce Supervised Reinforcement Asynchronous Learning (SuReAL). Learning uses both simulation and real environments without requiring autonomous flight in the physical environment during training, and combines supervised learning for predicting positions to visit and reinforcement learning for continuous control. We evaluate our approach on a natural language instruction-following task with a physical quadcopter, and demonstrate effective execution and exploration behavior.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 9; 2024: 4; 2025: 4; 2026: 3.

43 papers.

Sort:

By area:

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  2. 2021Waypoint Models for Instruction-guided Navigation in Continuous Environmentscited by 129
  3. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  4. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  5. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  6. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  7. 2022Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environmentscited by 86
  8. 2022Correcting Robot Plans with Natural Language Feedbackcited by 69
  9. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
  10. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  11. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  12. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
31 more
  1. 2022Aerial Vision-and-Dialog Navigationcited by 49
  2. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  3. 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 42
  4. 2021Vision-Language Navigation: A Survey and Taxonomycited by 34
  5. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  6. 2023Distilling Internet-Scale Vision-Language Models into Embodied Agentscited by 30
  7. 2023TypeFly: Flying Drones with Large Language Modelcited by 23
  8. 2021Continual Learning for Grounded Instruction Generation by Observing Human Following Behaviorcited by 19
  9. 2022Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Taskscited by 17
  10. 2024Policy Improvement using Language Feedback Modelscited by 16
  11. 2024VLN-Video: Utilizing Driving Videos for Outdoor Vision-and-Language Navigationcited by 14
  12. 2023RREx-BoT: Remote Referring Expressions with a Bag of Trickscited by 12
  13. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  14. 2021SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmarkcited by 11
  15. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  16. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  17. 2023Multi-Level Compositional Reasoning for Interactive Instruction Followingcited by 7
  18. 2025Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter Itcited by 6
  19. 2021Natural Language Specification of Reinforcement Learning Policies through Differentiable Decision Treescited by 4
  20. 2021Learning to Follow Language Instructions with Compositional Policiescited by 3
  21. 2022Entity Divider with Language Grounding in Multi-Agent Reinforcement Learningcited by 2
  22. 2022A Computational Interface to Translate Strategic Intent from Unstructured Language in a Low-Data Settingcited by 1
  23. 2023tagE: Enabling an Embodied Agent to Understand Human Instructionscited by 1
  24. 2024Provable Interactive Learning with Hindsight Instruction Feedbackcited by 1
  25. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  26. 2021Neural Abstructions: Abstractions that Support Construction for Grounded Language Learningcited by 0
  27. 2023Core Challenges in Embodied Vision-Language Planningcited by 0
  28. 2025Compositional Instruction Following with Language Models and Reinforcement Learningcited by 0
  29. 2025Spatial Language Likelihood Grounding Network for Bayesian Fusion of Human-Robot Observationscited by 0
  30. 2026GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Viewscited by 0
  31. 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0