mlsanityData to 7 Oct 2026

Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology

arXiv 2410.07087 ↗First posted 9 Oct 2024cs.CV cs.ROcited by 92

Not on the map. Why?

Abstract (arXiv)

Developing agents capable of navigating to a target location based on language instructions and visual information, known as vision-language navigation (VLN), has attracted widespread interest. Most research has focused on ground-based agents, while UAV-based VLN remains relatively underexplored. Recent efforts in UAV vision-language navigation predominantly adopt ground-based VLN settings, relying on predefined discrete action spaces and neglecting the inherent disparities in agent movement dynamics and the complexity of navigation tasks between ground and aerial environments. To address these disparities and challenges, we propose solutions from three perspectives: platform, benchmark, and methodology. To enable realistic UAV trajectory simulation in VLN tasks, we propose the OpenUAV platform, which features diverse environments, realistic flight control, and extensive algorithmic support. We further construct a target-oriented VLN dataset consisting of approximately 12k trajectories on this platform, serving as the first dataset specifically designed for realistic UAV VLN tasks. To tackle the challenges posed by complex aerial environments, we propose an assistant-guided UAV object search benchmark called UAV-Need-Help, which provides varying levels of guidance information to help UAVs better accomplish realistic VLN tasks. We also propose a UAV navigation LLM that, given multi-view images, task descriptions, and assistant instructions, leverages the multimodal understanding capabilities of the MLLM to jointly process visual and textual information, and performs hierarchical trajectory generation. The evaluation results of our method significantly outperform the baseline models, while there remains a considerable gap between our results and those achieved by human operators, underscoring the challenge presented by the UAV-Need-Help task.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

26 of 35 references.

References · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  6. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  7. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  9. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
  10. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 330
  11. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  12. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
14 more
  1. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  2. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  3. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  4. 2018xView: Objects in Context in Overhead Imagerycited by 145
  5. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  6. 2018Mapping Instructions to Actions in 3D Environments with Visual Goal Predictioncited by 101
  7. 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
  8. 2022Aerial Vision-and-Dialog Navigationcited by 48
  9. 2024CityNav: A Large-Scale Dataset for Real-World Aerial Navigationcited by 40
  10. 2018Mapping Navigation Instructions to Continuous Control Actions with Position-Visitation Predictioncited by 35
  11. 2020Multimodal Text Style Transfer for Outdoor Vision-and-Language Navigationcited by 22
  12. 2020AirSim Drone Racing Labcited by 15
  13. 2023A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMscited by 13
  14. 2019Cross-Lingual Vision-Language Navigationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 36; 2026: 53.

92 papers.

Sort:

By area:

  1. 2024Towards Long-Horizon Vision-Language Navigation: Platform, Benchmark and Methodcited by 66
  2. 2024VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selectioncited by 64
  3. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  4. 2025Openfly: A comprehensive platform for aerial vision-language navigationcited by 43
  5. 2025UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agentscited by 31
  6. 2025UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learningcited by 30
  7. 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 27
  8. 2025RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviourcited by 22
  9. 2025FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Modelscited by 22
  10. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 22
  11. 2025CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVscited by 21
  12. 2024Exploring Spatial Representation to Enhance LLM Reasoning in Aerial Vision-Language Navigationcited by 18
38 more
  1. 2025See, Point, Fly: A Learning-Free VLM Framework for Universal Unmanned Aerial Navigationcited by 18
  2. 2025SkyVLN: Vision-and-Language Navigation and NMPC Control for UAVs in Urban Environmentscited by 14
  3. 2025AeroDuo: Aerial Duo for UAV-based Vision and Language Navigationcited by 13
  4. 2025UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarioscited by 13
  5. 2025AirSim360: A Panoramic Simulation Platform within Drone Viewcited by 12
  6. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 12
  7. 2025Aerial World Model for Long-horizon Visual Generation and Navigation in 3D Spacecited by 11
  8. 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
  9. 2025Next-Generation LLM for UAV: From Natural Language to Autonomous Flightcited by 10
  10. 2025Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigationcited by 10
  11. 2026α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networkscited by 10
  12. 2025IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environmentscited by 9
  13. 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?cited by 9
  14. 2025UAV-CodeAgents: Scalable UAV Mission Planning via Multi-Agent ReAct and Vision-Language Reasoningcited by 7
  15. 2025FlySearch: Exploring how vision-language models explorecited by 7
  16. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 7
  17. 2026Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligencecited by 6
  18. 2025History-Enhanced Two-Stage Transformer for Aerial Vision-and-Language Navigationcited by 5
  19. 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructionscited by 5
  20. 2026APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigationcited by 5
  21. 2025Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agentcited by 4
  22. 2025AgriVLN: Vision-and-Language Navigation for Agricultural Robotscited by 4
  23. 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
  24. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  25. 2025FM-Planner: Foundation Model Guided Path Planning for Autonomous Drone Navigationcited by 3
  26. 2025GeoExplorer: Active Geo-localization with Curiosity-Driven Explorationcited by 3
  27. 2025SA-GCS: Semantic-Aware Gaussian Curriculum Scheduling for UAV Vision-Language Navigationcited by 3
  28. 2025TranSimHub:A Unified Air-Ground Simulation Platform for Multi-Modal Perception and Decision-Makingcited by 3
  29. 2026Fly0: Persistent Metric Anchoring for Zero-Shot Aerial Vision-Language Navigationcited by 3
  30. 2026HUGE-Bench: A Benchmark for High-Level UAV Vision-Language-Action Taskscited by 3
  31. 2026Are VLMs Lost Between Sky and Space? LinkS2Bench for UAV-Satellite Dynamic Cross-View Spatial Intelligencecited by 3
  32. 2026LookasideVLN: Direction-Aware Aerial Vision-and-Language Navigationcited by 3
  33. 2026Uni-LaViRA: Language-Vision-Robot Actions Translation for Unified Embodied Navigationcited by 3
  34. 2026DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Imagescited by 3
  35. 2025SoraNav: Adaptive UAV Task-Centric Navigation via Zeroshot VLM Reasoningcited by 2
  36. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  37. 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
  38. 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of Viewcited by 2

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
  2. 2025OpenVLN: Open-world Aerial Vision-Language Navigation
  3. 2026Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
  4. 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVs
  5. 2025Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding
  6. 2026AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild
  7. 2026SatNav: A Scalable Benchmark for Long-Horizon UAV Vision-Language Navigation from Satellite Imagery
  8. 2025LogisticsVLN: Vision-Language Navigation For Low-Altitude Terminal Delivery Based on Agentic UAVs