mlsanityData to 14 Oct 2026

FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models

arXiv 2505.12835 ↗First posted 19 May 2025cs.CL cs.CVcited by 22

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Unmanned Aerial Vehicle (UAV) Vision-and-Language Navigation (VLN) is vital for applications such as disaster response, logistics delivery, and urban inspection. However, existing methods often struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. To address these challenges, we propose FlightGPT, a novel UAV VLN framework built upon Vision-Language Models (VLMs) with powerful multimodal perception capabilities. We design a two-stage training pipeline: first, Supervised Fine-Tuning (SFT) using high-quality demonstrations to improve initialization and structured reasoning; then, Group Relative Policy Optimization (GRPO) algorithm, guided by a composite reward that considers goal accuracy, reasoning quality, and format compliance, to enhance generalization and adaptability. Furthermore, FlightGPT introduces a Chain-of-Thought (CoT)-based reasoning mechanism to improve decision interpretability. Extensive experiments on the city-scale dataset CityNav demonstrate that FlightGPT achieves state-of-the-art performance across all scenarios, with a 9.22% higher success rate than the strongest baseline in unseen environments. Our implementation is publicly available.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 27 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

13 of 27 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  5. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  6. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
  7. 2020A Recurrent Vision-and-Language BERT for Navigationcited by 302
  8. 2019Self-Monitoring Navigation Agent via Auxiliary Progress Estimationcited by 176
  9. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 121
  10. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 95
  11. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 61
  12. 2025UAV-VLA: Vision-Language-Action System for Large Scale Aerial Mission Generationcited by 28
1 more
  1. 2024Generating Content for HDR Deghosting from Frequency Viewcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 18.

22 papers.

Sort:

By area:

  1. 2025VLA-AN: An Efficient and Onboard Vision-Language-Action Framework for Aerial Navigation in Complex Environmentscited by 23
  2. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 14
  3. 2025UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVscited by 13
  4. 2026AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Controlcited by 11
  5. 2026NavDreamer: Video Models as Zero-Shot 3D Navigatorscited by 9
  6. 2025Aerial Vision-Language Navigation with a Unified Framework for Spatial, Temporal and Embodied Reasoningcited by 8
  7. 2026FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigationcited by 4
  8. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  9. 2026ViSA-Enhanced Aerial VLN: A Visual-Spatial Reasoning Enhanced Framework for Aerial Vision-Language Navigationcited by 2
  10. 2026Adaptive Robust Estimator for Multi-Agent Reinforcement Learningcited by 2
  11. 2026HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigationcited by 2
  12. 2026See-and-Reach: Precise Vision-Language Navigation for UAVs within the Field of Viewcited by 2
10 more
  1. 2026CLOSER-VLN: Closed-Loop Self-Verified Retrieval-Augmented Reasoning for Aerial Vision-Language Navigationcited by 2
  2. 2026Mitigating Error Accumulation in Continuous Navigation via Memory-Augmented Kalman Filteringcited by 1
  3. 2026SpatialFly: Implicit 3D Prior-Guided Visual Reparameterization for Continuous UAV Vision-and-Language Navigationcited by 1
  4. 2026Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPOcited by 1
  5. 2026ScoutVLA: UAV-Centric Active Perception via a Dual-Expert VLA Model for Open-World Embodied Question Answeringcited by 1
  6. 2026DreamFly: Causal Memory and Receding-Horizon Diffusion Planning for Aerial Vision-Language Navigationcited by 1
  7. 2026A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selectioncited by 0
  8. 2026No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigationcited by 0
  9. 2026RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigationcited by 0
  10. 2026Structured World-State Reasoning for Agentic Robotic Searchcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UAV-VL-R1: Generalizing Vision-Language Models via Supervised Fine-Tuning and Multi-Stage GRPO for UAV Visual Reasoning
  2. 2026Vision-Language Navigation for Aerial Robots: Towards the Era of Large Language Models
  3. 2025UAV-VLN: End-to-End Vision Language guided Navigation for UAVs
  4. 2026Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap
  5. 2025OpenVLN: Open-world Aerial Vision-Language Navigation
  6. 2026AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions
  7. 2025Grounded Vision-Language Navigation for UAVs with Open-Vocabulary Goal Understanding
  8. 2026FSD-VLN: Fast-Slow Dual-System Modeling for Aerial Long-Horizon Vision-Language Navigation