mlsanityData to 7 Oct 2026

UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobility

arXiv 2501.02341 ↗First posted 4 Jan 2025cs.RO cs.AIcited by 48

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Low-altitude mobility, exemplified by unmanned aerial vehicles (UAVs), has introduced transformative advancements across various domains, like transportation, logistics, and agriculture. Leveraging flexible perspectives and rapid maneuverability, UAVs extend traditional systems' perception and action capabilities, garnering widespread attention from academia and industry. However, current UAV operations primarily depend on human control, with only limited autonomy in simple scenarios, and lack the intelligence and adaptability needed for more complex environments and tasks. The emergence of large language models (LLMs) demonstrates remarkable problem-solving and generalization capabilities, offering a promising pathway for advancing UAV intelligence. This paper explores the integration of LLMs and UAVs, beginning with an overview of UAV systems' fundamental components and functionalities, followed by an overview of the state-of-the-art in LLM technology. Subsequently, it systematically highlights the multimodal data resources available for UAVs, which provide critical support for training and evaluation. Furthermore, it categorizes and analyzes key tasks and application scenarios where UAVs and LLMs converge. Finally, a reference roadmap towards agentic UAVs is proposed, aiming to enable UAVs to achieve agentic intelligence through autonomous perception, memory, reasoning, and tool utilization. Related resources are available at https://github.com/Hub-Tian/UAVs_Meet_LLMs.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 178 of its 413 references that are in the map. First posted on arXiv 4 Jan 2025.

Builds on

178 of 413 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environment12 Oct
  2. 2024AeroVerse: UAV-Agent Benchmark Suite for Simulating, Pre-training, Finetuning, and Evaluating Aerospace Embodied World Models28 Aug
  3. 2024HYDRA: A Hyper Agent for Dynamic Compositional Visual Reasoning19 Mar
  4. 2024Agent3D-Zero: An Agent for Zero-shot 3D Understanding18 Mar

Other references · 174

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2024The Llama 3 Herd of Modelscited by 14,192
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2023Segment Anythingcited by 8,158
  12. 2023Visual Instruction Tuningcited by 7,792
38 more
  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  12. 2023A Survey of Large Language Modelscited by 2,537
  13. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  14. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  15. 2024Qwen2 Technical Reportcited by 2,076
  16. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  17. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  18. 2024Depth Anything V2cited by 1,385
  19. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  20. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  21. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  22. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  23. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  24. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  25. 2021Grounded Language-Image Pre-trainingcited by 1,147
  26. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  27. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  28. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  29. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  30. 2023A Survey on Multimodal Large Language Modelscited by 898
  31. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  32. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  33. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  34. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  35. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  36. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  37. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  38. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677

50 most cited of 174 All 174 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 29; 2026: 19.

48 papers.

Sort:

By area:

  1. 2025RaceVLA: VLA-based Racing Drone Navigation with Human-like Behaviourcited by 22
  2. 2025CognitiveDrone: A VLA Model and Evaluation Benchmark for Real-Time Cognitive Task Solving and Reasoning in UAVscited by 21
  3. 2025SkyVLN: Vision-and-Language Navigation and NMPC Control for UAVs in Urban Environmentscited by 14
  4. 2025UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVscited by 13
  5. 2025LLM-Enabled In-Context Learning for Data Collection Scheduling in UAV-assisted Sensor Networkscited by 12
  6. 2025BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVscited by 10
  7. 2025Where, What, Why: Towards Explainable Driver Attention Predictioncited by 10
  8. 2025Next-Generation LLM for UAV: From Natural Language to Autonomous Flightcited by 10
  9. 2025CHAI: Command Hijacking against embodied AIcited by 9
  10. 2025Agentic UAVs: LLM-Driven Autonomy with Integrated Tool-Calling and Cognitive Reasoningcited by 7
  11. 2026APEX: A Decoupled Memory-based Explorer for Asynchronous Aerial Object Goal Navigationcited by 5
  12. 2025AirVista-II: An Agentic System for Embodied UAVs Toward Dynamic Scene Semantic Understandingcited by 4
36 more
  1. 2025Taking Flight with Dialogue: Enabling Natural Language Control for PX4-based Drone Agentcited by 4
  2. 2025FRSICL: LLM-Enabled In-Context Learning Flight Resource Allocation for Fresh Data Collection in UAV-Assisted Wildfire Monitoringcited by 4
  3. 2026UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understandingcited by 4
  4. 2026UAV-Track VLA: Embodied Aerial Tracking via Vision-Language-Action Modelscited by 4
  5. 2026ESARBench: A Benchmark for Agentic UAV Embodied Search and Rescuecited by 4
  6. 2026Say the Mission, Execute the Swarm: Agent-Enhanced LLM Reasoning in the Web-of-Dronescited by 4
  7. 2025Recent Advances in Transformer and Large Language Models for UAV Applicationscited by 3
  8. 2025A Hierarchical Agentic Framework for Autonomous Drone-Based Visual Inspectioncited by 3
  9. 2026DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Imagescited by 3
  10. 2025CoordField: Coordination Field for Agentic UAV Task Allocation In Low-altitude Urban Scenarioscited by 2
  11. 2025Agentic Satellite-Augmented Low-Altitude Economy and Terrestrial Networks: A Survey on Generative Approachescited by 2
  12. 2025TACOS: Task Agnostic COordinator of a multi-drone Systemcited by 2
  13. 2025Is Intermediate Fusion All You Need for UAV-based Collaborative Perception?cited by 1
  14. 2025Autonomous Navigation of Cloud-Controlled Quadcopters in Confined Spaces Using Multi-Modal Perception and LLM-Driven High Semantic Reasoningcited by 1
  15. 2025TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrievalcited by 1
  16. 2025All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehiclescited by 1
  17. 2025AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Modelscited by 1
  18. 2026PINA: Prompt Injection Attack against Navigation Agentscited by 1
  19. 2026Parse, Search, and Confirmation: Training-Free Aerial Vision-and-Dialog Navigation with Chain-of-Thought Reasoning and Structured Spatial Memorycited by 1
  20. 2026CosFly-VLA: A Spatially Aware Vision-Language-Action Model for UAV Trackingcited by 1
  21. 2026AeroDPO: Unleashing Lightweight UAV Navigation with High-Fidelity Perception and Automated Preference Optimizationcited by 1
  22. 2025Mathematical Reasoning for Unmanned Aerial Vehicles: A RAG-Based Approach for Complex Arithmetic Reasoningcited by 0
  23. 2025Tactical Decision for Multi-UGV Confrontation with a Vision-Language Model-Based Commandercited by 0
  24. 2025Semantic-Aware Edge Intelligence for UAV Handover in 6G Networkscited by 0
  25. 2025Cognition Envelopes for Bounded Decision Making in Autonomous UAS Operationscited by 0
  26. 2025PrivLLMSwarm: Privacy-Preserving LLM-Driven UAV Swarms for Secure IoT Surveillancecited by 0
  27. 2025LEO-RobotAgent: A General-purpose Robotic Agent for Language-driven Embodied Operatorcited by 0
  28. 2026A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selectioncited by 0
  29. 2026Voice-Driven Semantic Perception for UAV-Assisted Emergency Networkscited by 0
  30. 2026See&Say: Vision Language Guided Safe Zone Detection for Autonomous Package Delivery Dronescited by 0
  31. 2026PEACE: A Planner-Executor Agent with Constraint Enforcement for UAVscited by 0
  32. 2026CAMF-Det: Closure-Aware Multimodal Fusion for LiDAR-Camera 3D Object Detection on UAV Platformscited by 0
  33. 2026Beyond Attack Success Rate: Examining Trigger Leakage in Vision-Language Agentic Systemscited by 0
  34. 2026RefGlass-GS: A UAV-Enabled Fusion Framework for Photorealistic, Semantic and Interactive Digitization of Reflective Glass Facades via Gaussian Splattingcited by 0
  35. 2026RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Controlcited by 0
  36. 2026From Semantic Grounding to Decision Optimization: A Unified Framework for Long-Horizon UAV Vision-Language Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Large Language Model-Assisted UAV Operations and Communications: A Multifaceted Survey and Tutorial
  2. 2025When Large Language Models Meet UAV Projects: An Empirical Study from Developers' Perspective
  3. 2025General-Purpose Aerial Intelligent Agents Empowered by Large Language Models
  4. 2025Large Language Model-Driven Closed-Loop UAV Operation with Semantic Observations
  5. 2025Multimodal Large Language Models-Enabled UAV Swarm: Towards Efficient and Intelligent Autonomous Aerial Systems
  6. 2026α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks
  7. 2025MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?
  8. 2026LAPF: LLM-Agent-Based Path Finder Using the UAVScenes Dataset