mlsanityData to 14 Oct 2026

Aerial Vision-and-Dialog Navigation

arXiv 2205.12219 ↗First posted 24 May 2022cs.CV cs.AI cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

The ability to converse with humans and follow natural language commands is crucial for intelligent unmanned aerial vehicles (a.k.a. drones). It can relieve people's burden of holding a controller all the time, allow multitasking, and make drone control more accessible for people with disabilities or with their hands occupied. To this end, we introduce Aerial Vision-and-Dialog Navigation (AVDN), to navigate a drone via natural language conversation. We build a drone simulator with a continuous photorealistic environment and collect a new AVDN dataset of over 3k recorded navigation trajectories with asynchronous human-human dialogs between commanders and followers. The commander provides initial navigation instruction and further guidance by request, while the follower navigates the drone in the simulator and asks questions when needed. During data collection, followers' attention on the drone's visual observation is also recorded. Based on the AVDN dataset, we study the tasks of aerial navigation from (full) dialog history and propose an effective Human Attention Aided Transformer model (HAA-Transformer), which learns to predict both navigation waypoints and human attention.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

25 of 34 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2018YOLOv3: An Incremental Improvementcited by 2,659
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  7. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  10. 2019Vision-and-Dialog Navigationcited by 250
  11. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  12. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
13 more
  1. 2021Episodic Transformer for Vision-and-Language Navigationcited by 156
  2. 2018xView: Objects in Context in Overhead Imagerycited by 146
  3. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  4. 2019Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learningcited by 100
  5. 2018Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Interventioncited by 77
  6. 2022DialFRED: Dialogue-Enabled Agents for Embodied Instruction Followingcited by 63
  7. 2020AU-AIR: A Multi-modal Unmanned Aerial Vehicle Dataset for Low Altitude Traffic Surveillancecited by 56
  8. 2019Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flightcited by 43
  9. 2019Generalization through Simulation: Integrating Simulated and Real Data into Deep Reinforcement Learning for Vision-Based Autonomous Flightcited by 28
  10. 2017Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awarenesscited by 19
  11. 2019Understanding and Visualizing Deep Visual Saliency Modelscited by 13
  12. 2020Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigatorcited by 5
  13. 2018Learning Transferable UAV for Forest Visual Perceptioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 3; 2025: 19; 2026: 22.

49 papers.

Sort:

By area:

  1. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  2. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  3. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 46
  4. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 42
  5. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  6. 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
  7. 2025RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviourcited by 23
  8. 2025CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVscited by 22
  9. 2025RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenescited by 19
  10. 2025See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigationcited by 18
  11. 2025Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigationcited by 17
  12. 2025SkyVLN: Vision-and-Language Navigation and NMPC Control for UAVs in Urban Environmentscited by 14
37 more
  1. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
  2. 2025Aerial Vision-and-Language Navigation with Grid-based View Selection and Map Constructioncited by 12
  3. 2025AirSim360: A Panoramic Simulation Platform within Drone Viewcited by 12
  4. 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
  5. 2025General-Purpose Aerial Intelligent Agents Empowered by Large Language Modelscited by 9
  6. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
  7. 2025UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoningcited by 7
  8. 2023R2H: Building Multimodal Navigation Helpers that Respond to Help Requestscited by 5
  9. 2025CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigationcited by 5
  10. 2025Generating Vision-Language Navigation Instructions Incorporated Fine-Grained Alignment Annotationscited by 5
  11. 2025Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agentcited by 4
  12. 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
  13. 2025GeoExplorer: Active Geo-localization with Curiosity-Driven Explorationcited by 3
  14. 2025DialNav: Multi-turn Dialog Navigation with a Remote Guidecited by 3
  15. 2026LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigationcited by 3
  16. 2026Instance-level Visual Active Tracking with Occlusion-Aware Planningcited by 3
  17. 2026FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigationcited by 3
  18. 2023Target-Grounded Graph-Aware Transformer for Aerial Vision-and-Dialog Navigationcited by 2
  19. 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
  20. 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of Viewcited by 2
  21. 2026Knowing the Self, Understanding the World: A Dual-Cognition Benchmark for UAV Spatio-temporal Reasoning with MLLMscited by 2
  22. 2023Paparazzi: A Deep Dive into the Capabilities of Language and Vision Models for Grounding Viewpoint Descriptionscited by 1
  23. 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
  24. 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
  25. 2026A Comprehensive Survey and Systematic Real-World Evaluation of Embodied Vision-and-Language Navigationcited by 1
  26. 2026Parse, Search, and Confirmation: Training-Free Aerial Vision-and-Dialog Navigation with Chain-of-Thought Reasoning and Structured Spatial Memorycited by 1
  27. 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigationcited by 1
  28. 2023Multi-model fusion for Aerial Vision and Dialog Navigation based on human attention aidscited by 0
  29. 2026Hierarchical Dual-Change Collaborative Learning for UAV Scene Change Captioningcited by 0
  30. 2026How Far Are Large Multimodal Models from Human-Level Spatial Action? A Benchmark for Goal-Oriented Embodied Navigation in Urban Airspacecited by 0
  31. 2026Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checkercited by 0
  32. 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigationcited by 0
  33. 2026Zero-Shot Mission-Level Evaluation for Aerial MLLM Agentscited by 0
  34. 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
  35. 2026Structured World-State Reasoning for Agentic Robotic Searchcited by 0
  36. 2026ForeFly: A Dual-Horizon World Action Model for Aerial Vision-Language Navigationcited by 0
  37. 2026WareFly-VLA: A Vision-Language-Action Framework for UAV Navigation and Human Tracking in Smart Warehousescited by 0