FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Unmanned Aerial Vehicle (UAV) Vision-and-Language Navigation (VLN) is vital for applications such as disaster response, logistics delivery, and urban inspection. However, existing methods often struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. To address these challenges, we propose FlightGPT, a novel UAV VLN framework built upon Vision-Language Models (VLMs) with powerful multimodal perception capabilities. We design a two-stage training pipeline: first, Supervised Fine-Tuning (SFT) using high-quality demonstrations to improve initialization and structured reasoning; then, Group Relative Policy Optimization (GRPO) algorithm, guided by a composite reward that considers goal accuracy, reasoning quality, and format compliance, to enhance generalization and adaptability. Furthermore, FlightGPT introduces a Chain-of-Thought (CoT)-based reasoning mechanism to improve decision interpretability. Extensive experiments on the city-scale dataset CityNav demonstrate that FlightGPT achieves state-of-the-art performance across all scenarios, with a 9.22% higher success rate than the strongest baseline in unseen environments. Our implementation is publicly available.
Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 27 references that are in the map. First posted on arXiv 19 May 2025.
Builds on
13 of 27 references.
References · 13
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
- 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
- 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
- 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
- 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 61
- 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
1 more
- 2024Generating Content for HDR Deghosting from Frequency Viewcited by 14
Built on by
Citing papers per year: 2025: 4; 2026: 18.
22 papers.
Sort:
By area:
- 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 23
- 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
- 2025UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVscited by 13
- 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
- 2026NavDreamer: Video Models as Zero-Shot 3D Navigatorscited by 9
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
- 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
- 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
- 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
- 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learningcited by 2
- 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
- 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of Viewcited by 2
10 more
- 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
- 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
- 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
- 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answeringcited by 1
- 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigationcited by 1
- 2026A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selectioncited by 0
- 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigationcited by 0
- 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
- 2026Structured World-State Reasoning for Agentic Robotic Searchcited by 0
- 2026Structured World-State Reasoning for Agentic Robotic Search20 Sep 2026
- 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation12 Aug 2026
- 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation10 Aug 2026
- 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigation21 Jul 2026
- 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigation24 Jun 2026
- 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View18 Jun 2026
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering9 Jun 2026
- 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO1 Jun 2026
- 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescue2 May 2026
- 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation17 Apr 2026
- 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation10 Apr 2026
- 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learning23 Mar 2026
10 more
- 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation22 Mar 2026
- 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control15 Mar 2026
- 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation9 Mar 2026
- 2026NavDreamer: Video Models as Zero-Shot 3D Navigators10 Feb 2026
- 2026A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selection1 Feb 2026
- 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering30 Jan 2026
- 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration26 Dec 2025
- 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments17 Dec 2025
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning9 Dec 2025
- 2025UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs8 Jun 2025
- 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
- 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learningcited by 2
- 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
- 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigationcited by 0
- 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigation21 Jul 2026
- 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO1 Jun 2026
- 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learning23 Mar 2026
- 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigation9 Mar 2026
- 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 23
- 2026NavDreamer: Video Models as Zero-Shot 3D Navigatorscited by 9
- 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
- 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation10 Aug 2026
- 2026NavDreamer: Video Models as Zero-Shot 3D Navigators10 Feb 2026
- 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environments17 Dec 2025
- 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
- 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
- 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
- 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
- 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of Viewcited by 2
- 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
- 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answeringcited by 1
- 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigationcited by 1
- 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigation12 Aug 2026
- 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of View18 Jun 2026
- 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answering9 Jun 2026
- 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation17 Apr 2026
- 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation10 Apr 2026
- 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigation22 Mar 2026
- 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control15 Mar 2026
- 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filtering30 Jan 2026
- 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integration26 Dec 2025
- 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoning9 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning
- 2026Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
- 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVs
- 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
- 2025OpenVLN: Open-world Aerial Vision-Language Navigation
- 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions
- 2025Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding
- 2026FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigation