# UAV-VLN: End-to-End Vision Language guided Navigation for UAVs

arXiv: https://arxiv.org/abs/2504.21432 · HTML: https://mlsanity.com/paper/2504.21432/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 34 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 19 of 19)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2004.02857/) (2020)
- [Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding](https://mlsanity.com/paper/2010.07954/) (2020)
- [Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents](https://mlsanity.com/paper/2302.01560/) (2023)
- [Vision-and-Dialog Navigation](https://mlsanity.com/paper/1907.04957/) (2019)
- [Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments](https://mlsanity.com/paper/1811.12354/) (2018)
- [TEACh: Task-driven Embodied Agents that Chat](https://mlsanity.com/paper/2110.00534/) (2021)
- [L3MVN: Leveraging Large Language Models for Visual Target Navigation](https://mlsanity.com/paper/2304.05501/) (2023)
- [Mapping Instructions to Actions in 3D Environments with Visual Goal Prediction](https://mlsanity.com/paper/1809.00786/) (2018)
- [GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation](https://mlsanity.com/paper/2404.06609/) (2024)
- [Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention](https://mlsanity.com/paper/1812.04155/) (2018)
- [Explore until Confident: Efficient Exploration for Embodied Question Answering](https://mlsanity.com/paper/2403.15941/) (2024)
- [Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challenges](https://mlsanity.com/paper/1805.00881/) (2018)

## Built on by (most cited first, 11 of 11)

- [LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration](https://mlsanity.com/paper/2512.22010/) (2025)
- [Agentic UAVs: LLM-Driven Autonomy with Integrated Tool-Calling and Cognitive Reasoning](https://mlsanity.com/paper/2509.13352/) (2025)
- [Asking like Socrates: Socrates helps VLMs understand remote sensing images](https://mlsanity.com/paper/2511.22396/) (2025)
- [Self in Space: Benchmarking Self-Awareness and Spatial Cognition in UAV Embodied Intelligence](https://mlsanity.com/paper/2607.12477/) (2026)
- [Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial](https://mlsanity.com/paper/2602.19534/) (2026)
- [FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation](https://mlsanity.com/paper/2604.16298/) (2026)
- [Embodied4C: Measuring What Matters for Embodied Vision-Language Navigation](https://mlsanity.com/paper/2512.18028/) (2025)
- [Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO](https://mlsanity.com/paper/2606.02313/) (2026)
- [From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigation](https://mlsanity.com/paper/2608.09564/) (2026)
- [VLN-Pilot: Large Vision-Language Model as an Autonomous Indoor Drone Operator](https://mlsanity.com/paper/2602.05552/) (2026)
- [Vision Language Action (VLA) Models for Unmanned Aerial Robotics and Bimanual Manipulation: A Review](https://mlsanity.com/paper/2607.06706/) (2026)

## Abstract (arXiv)

A core challenge in AI-guided autonomy is enabling agents to navigate realistically and effectively in previously unseen environments based on natural language commands. We propose UAV-VLN, a novel end-to-end Vision-Language Navigation (VLN) framework for Unmanned Aerial Vehicles (UAVs) that seamlessly integrates Large Language Models (LLMs) with visual perception to facilitate human-interactive navigation. Our system interprets free-form natural language instructions, grounds them into visual observations, and plans feasible aerial trajectories in diverse environments. UAV-VLN leverages the common-sense reasoning capabilities of LLMs to parse high-level semantic goals, while a vision model detects and localizes semantically relevant objects in the environment. By fusing these modalities, the UAV can reason about spatial relationships, disambiguate references in human instructions, and plan context-aware behaviors with minimal task-specific supervision. To ensure robust and interpretable decision-making, the framework includes a cross-modal grounding mechanism that aligns linguistic intent with visual context. We evaluate UAV-VLN across diverse indoor and outdoor navigation scenarios, demonstrating its ability to generalize to novel instructions and environments with minimal task-specific training. Our results show significant improvements in instruction-following accuracy and trajectory efficiency, highlighting the potential of LLM-driven vision-language interfaces for safe, intuitive, and generalizable UAV autonomy.