mlsanityData to 7 Oct 2026

Next-Generation LLM for UAV: From Natural Language to Autonomous Flight

arXiv 2510.21739 ↗First posted 2 Oct 2025cs.RO cs.AI cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

With the rapid advancement of Large Language Models (LLMs), their capabilities in various automation domains, particularly Unmanned Aerial Vehicle (UAV) operations, have garnered increasing attention. Current research remains predominantly constrained to small-scale UAV applications, with most studies focusing on isolated components such as path planning for toy drones, while lacking comprehensive investigation of medium- and long-range UAV systems in real-world operational contexts. Larger UAV platforms introduce distinct challenges, including stringent requirements for airport-based take-off and landing procedures, adherence to complex regulatory frameworks, and specialized operational capabilities with elevated mission expectations. This position paper presents the Next-Generation LLM for UAV (NeLV) system -- a comprehensive demonstration and automation roadmap for integrating LLMs into multi-scale UAV operations. The NeLV system processes natural language instructions to orchestrate short-, medium-, and long-range UAV missions through five key technical components: (i) LLM-as-Parser for instruction interpretation, (ii) Route Planner for Points of Interest (POI) determination, (iii) Path Planner for waypoint generation, (iv) Control Platform for executable trajectory implementation, and (v) UAV monitoring. We demonstrate the system's feasibility through three representative use cases spanning different operational scales: multi-UAV patrol, multi-POI delivery, and multi-hop relocation. Beyond the current implementation, we establish a five-level automation taxonomy that charts the evolution from current LLM-as-Parser capabilities (Level 1) to fully autonomous LLM-as-Autopilot systems (Level 5), identifying technical prerequisites and research challenges at each stage.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 25 of its 58 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

25 of 58 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  3. 2023AerialVLN: Vision-and-Language Navigation for UAVscited by 117
  4. 2024Towards Realistic UAV Vision-Language Navigation: Platform, Benchmark, and Methodologycited by 92
  5. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 67
  6. 2018Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challengescited by 49
  7. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 48
  8. 2023Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matchingcited by 40
  9. 2024Large Language Models for UAVs: Current State and Pathways to the Futurecited by 33
  10. 2022Machine Learning-Aided Operations and Communications of Unmanned Aerial Vehicles: A Contemporary Surveycited by 27
  11. 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Modelscited by 24
  12. 2023TypeFly: Flying Drones with Large Language Modelcited by 23
13 more
  1. 2025Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directionscited by 23
  2. 2023REAL: Resilience and Adaptation using Large Language Models on Autonomous Aerial Robotscited by 19
  3. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHFcited by 18
  4. 2024Generative AI for Unmanned Vehicle Swarms: Challenges, Applications and Opportunitiescited by 14
  5. 2023A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMscited by 13
  6. 2024Generative AI for Advanced UAV Networkingcited by 13
  7. 2025Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversationscited by 13
  8. 2022Evolutionary Multi-Objective Reinforcement Learning Based Trajectory Control and Task Offloading in UAV-Assisted Mobile Edge Computingcited by 10
  9. 2024Multi-objective Optimization for Multi-UAV-assisted Mobile Edge Computingcited by 8
  10. 2024Trustworthy Human-AI Collaboration: Reinforcement Learning with Human Feedback and Physics Knowledge for Safe Autonomous Drivingcited by 8
  11. 2025LLMAP: LLM-Assisted Multi-Objective Route Planning with User Preferencescited by 5
  12. 2025Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Trainingcited by 4
  13. 2024CLIPSwarm: Generating Drone Shows from Text Prompts with Vision-Language Modelscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 8.

10 papers.

  1. 2025UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarioscited by 13
  2. 2025LongFly: Long-Horizon UAV Vision-and-Language Navigation with Spatiotemporal Context Integrationcited by 12
  3. 2026α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networkscited by 10
  4. 2026Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorialcited by 4
  5. 2026α3-SecBench: A Large-Scale Evaluation Suite of Security, Resilience, and Trust for LLM-based UAV Agents over 6G Networkscited by 2
  6. 2026An LLM-Agnostic, MAVLink-Based Drone Command and Control Interface and Agentic Harness Using the Model Context Protocolcited by 1
  7. 2026Fine-Tuning Large Language Models for Cooperative Tactical Deconfliction of Small Unmanned Aerial Systemscited by 1
  8. 2026Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Modelscited by 1
  9. 2026Bridging Network Fragmentation: A Semantic-Augmented DRL Framework for UAV-aided VANETscited by 0
  10. 2026RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Controlcited by 0