# Chat with UAV -- Human-UAV Interaction Based on Large Language Models

arXiv: https://arxiv.org/abs/2512.08145 · HTML: https://mlsanity.com/paper/2512.08145/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 9 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [CLAD: A realistic Continual Learning benchmark for Autonomous Driving](https://mlsanity.com/paper/2210.03482/) (2022)
- [LLM as A Robotic Brain: Unifying Egocentric Memory and Control](https://mlsanity.com/paper/2304.09349/) (2023)
- [Large Language Models for UAVs: Current State and Pathways to the Future](https://mlsanity.com/paper/2405.01745/) (2024)
- [ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning](https://mlsanity.com/paper/2406.19741/) (2024)
- [Empowering Embodied Manipulation: A Bimanual-Mobile Robot Manipulation Dataset for Household Tasks](https://mlsanity.com/paper/2405.18860/) (2024)
- [LLM-based Robot Task Planning with Exceptional Handling for General Purpose Service Robots](https://mlsanity.com/paper/2405.15646/) (2024)
- [Generative AI for Advanced UAV Networking](https://mlsanity.com/paper/2404.10556/) (2024)
- [FlockGPT: Guiding UAV Flocking with Linguistic Orchestration](https://mlsanity.com/paper/2405.05872/) (2024)
- [Robust Trajectory and Resource Optimization for Communication-assisted UAV SAR Sensing](https://mlsanity.com/paper/2404.01195/) (2024)

## Built on by (most cited first, 1 of 1)

- [An LLM-Agnostic, MAVLink-Based Drone Command and Control Interface and Agentic Harness Using the Model Context Protocol](https://mlsanity.com/paper/2601.15486/) (2026)

## Abstract (arXiv)

The future of UAV interaction systems is evolving from engineer-driven to user-driven, aiming to replace traditional predefined Human-UAV Interaction designs. This shift focuses on enabling more personalized task planning and design, thereby achieving a higher quality of interaction experience and greater flexibility, which can be used in many fileds, such as agriculture, aerial photography, logistics, and environmental monitoring. However, due to the lack of a common language between users and the UAVs, such interactions are often difficult to be achieved. The developments of Large Language Models possess the ability to understand nature languages and Robots' (UAVs') behaviors, marking the possibility of personalized Human-UAV Interaction. Recently, some HUI frameworks based on LLMs have been proposed, but they commonly suffer from difficulties in mixed task planning and execution, leading to low adaptability in complex scenarios. In this paper, we propose a novel dual-agent HUI framework. This framework constructs two independent LLM agents (a task planning agent, and an execution agent) and applies different Prompt Engineering to separately handle the understanding, planning, and execution of tasks. To verify the effectiveness and performance of the framework, we have built a task database covering four typical application scenarios of UAVs and quantified the performance of the HUI framework using three independent metrics. Meanwhile different LLM models are selected to control the UAVs with compared performance. Our user study experimental results demonstrate that the framework improves the smoothness of HUI and the flexibility of task execution in the tasks scenario we set up, effectively meeting users' personalized needs.