# FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models

arXiv: https://arxiv.org/abs/2505.12835 · HTML: https://mlsanity.com/paper/2505.12835/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 13 of 13)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [Speaker-Follower Models for Vision-and-Language Navigation](https://mlsanity.com/paper/1806.02724/) (2018)
- [A Recurrent Vision-and-Language BERT for Navigation](https://mlsanity.com/paper/2011.13922/) (2020)
- [Self-Monitoring Navigation Agent via Auxiliary Progress Estimation](https://mlsanity.com/paper/1901.03035/) (2019)
- [AerialVLN: Vision-and-Language Navigation for UAVs](https://mlsanity.com/paper/2308.06735/) (2023)
- [Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodology](https://mlsanity.com/paper/2410.07087/) (2024)
- [Learning Generalizable Manipulation Policies with Object-Centric 3D Representations](https://mlsanity.com/paper/2310.14386/) (2023)
- [UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generation](https://mlsanity.com/paper/2501.05014/) (2025)
- [Generating Content for HDR Deghosting from Frequency View](https://mlsanity.com/paper/2404.00849/) (2024)

## Built on by (most cited first, 22 of 22)

- [VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments](https://mlsanity.com/paper/2512.15258/) (2025)
- [LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration](https://mlsanity.com/paper/2512.22010/) (2025)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control](https://mlsanity.com/paper/2603.14363/) (2026)
- [NavDreamer: Video Models as Zero-Shot 3D Navigators](https://mlsanity.com/paper/2602.09765/) (2026)
- [Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning](https://mlsanity.com/paper/2512.08639/) (2025)
- [FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation](https://mlsanity.com/paper/2604.16298/) (2026)
- [ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue](https://mlsanity.com/paper/2605.01371/) (2026)
- [ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2603.08007/) (2026)
- [Adaptive Robust Estimator for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2603.21574/) (2026)
- [HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation](https://mlsanity.com/paper/2604.08883/) (2026)
- [See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View](https://mlsanity.com/paper/2606.20045/) (2026)
- [CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2606.28397/) (2026)
- [Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering](https://mlsanity.com/paper/2602.11183/) (2026)
- [SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation](https://mlsanity.com/paper/2603.21046/) (2026)
- [Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO](https://mlsanity.com/paper/2606.02313/) (2026)
- [ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering](https://mlsanity.com/paper/2606.14772/) (2026)
- [DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2608.12308/) (2026)
- [A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selection](https://mlsanity.com/paper/2602.01163/) (2026)
- [No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigation](https://mlsanity.com/paper/2607.19288/) (2026)
- [RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation](https://mlsanity.com/paper/2608.09467/) (2026)
- [Structured World-State Reasoning for Agentic Robotic Search](https://mlsanity.com/paper/2609.23841/) (2026)

## Abstract (arXiv)

Unmanned Aerial Vehicle (UAV) Vision-and-Language Navigation (VLN) is vital for applications such as disaster response, logistics delivery, and urban inspection. However, existing methods often struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. To address these challenges, we propose FlightGPT, a novel UAV VLN framework built upon Vision-Language Models (VLMs) with powerful multimodal perception capabilities. We design a two-stage training pipeline: first, Supervised Fine-Tuning (SFT) using high-quality demonstrations to improve initialization and structured reasoning; then, Group Relative Policy Optimization (GRPO) algorithm, guided by a composite reward that considers goal accuracy, reasoning quality, and format compliance, to enhance generalization and adaptability. Furthermore, FlightGPT introduces a Chain-of-Thought (CoT)-based reasoning mechanism to improve decision interpretability. Extensive experiments on the city-scale dataset CityNav demonstrate that FlightGPT achieves state-of-the-art performance across all scenarios, with a 9.22% higher success rate than the strongest baseline in unseen environments. Our implementation is publicly available.