# Agentic AI for Safety-critical Multi-drone Systems: Challenges and Opportunities

arXiv: https://arxiv.org/abs/2608.21444 · HTML: https://mlsanity.com/paper/2608.21444/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 7 of its 29 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 7 of 7)

- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [Large Language Models for Robotics: Opportunities, Challenges, and Perspectives](https://mlsanity.com/paper/2401.04334/) (2024)
- [Towards autonomous system: flexible modular production system enhanced with large language model agents](https://mlsanity.com/paper/2304.14721/) (2023)
- [DisasterResponseGPT: Large Language Models for Accelerated Plan of Action Development in Disaster Response Scenarios](https://mlsanity.com/paper/2306.17271/) (2023)
- [UAVs Meet Agentic AI: A Multidomain Survey of Autonomous Aerial Intelligence and Agentic UAVs](https://mlsanity.com/paper/2506.08045/) (2025)
- [Cognitive Guardrails for Open-World Decision Making in Autonomous Drone Swarms](https://mlsanity.com/paper/2505.23576/) (2025)
- [Human-LLM Synergy in Context-Aware Adaptive Architecture for Scalable Drone Swarm Operation](https://mlsanity.com/paper/2509.05355/) (2025)

## Abstract (arXiv)

Multi-drone systems are increasingly positioned for safety-critical missions such as search and rescue (SAR) and critical infrastructure monitoring. Yet, real-world adoption remains constrained not only by autonomy performance, but by the difficulty of integrating agentic behavior into professional work: operators must understand, trust, and govern automation under uncertainty, time pressure, and accountability. This position paper synthesizes the ambitions and lessons from two ongoing efforts: NAMUR, which explores LLM-supported robot control in SAR and firefighting contexts, and PERSIST, which explores persistent drone operations for monitoring and security at critical infrastructure sites. We argue that agentic AI should be approached as a socio-technical design problem, where interfaces, oversight mechanisms, and evaluation practices are as critical as algorithms. We outline a human-centered, participatory, and iterative research approach aimed at uncovering stakeholder needs, shaping agent capabilities through successive prototypes, and producing transferable proof-of-concept systems and evaluation strategies for other safety-critical contexts.