# α3-SecBench: A Large-Scale Evaluation Suite of Security, Resilience, and Trust for LLM-based UAV Agents over 6G Networks

arXiv: https://arxiv.org/abs/2601.18754 · HTML: https://mlsanity.com/paper/2601.18754/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 26 Jan 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 17 of 17)

- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Pushing Large Language Models to the 6G Edge: Vision, Challenges, and Opportunities](https://mlsanity.com/paper/2309.16739/) (2023)
- [UAV-ON: A Benchmark for Open-World Object Goal Navigation with Aerial Agents](https://mlsanity.com/paper/2508.00288/) (2025)
- [AirCopBench: A Benchmark for Multi-drone Collaborative Embodied Perception and Reasoning](https://mlsanity.com/paper/2511.11025/) (2025)
- [UAVBench: An Open Benchmark Dataset for Autonomous and Agentic AI UAV Systems via LLM-Generated Flight Scenarios](https://mlsanity.com/paper/2511.11252/) (2025)
- [BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs](https://mlsanity.com/paper/2505.18229/) (2025)
- [Next-Generation LLM for UAV: From Natural Language to Autonomous Flight](https://mlsanity.com/paper/2510.21739/) (2025)
- [Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation](https://mlsanity.com/paper/2511.13269/) (2025)
- [MM-UAVBench: How Well Do Multimodal Large Language Models See, Think, and Plan in Low-Altitude UAV Scenarios?](https://mlsanity.com/paper/2512.23219/) (2025)
- [α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks](https://mlsanity.com/paper/2601.03281/) (2026)
- [LLM-Land: Large Language Models for Context-Aware Drone Landing](https://mlsanity.com/paper/2505.06399/) (2025)
- [RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms](https://mlsanity.com/paper/2507.01378/) (2025)
- [AirNav: A Large-Scale UAV Vision-and-Language Navigation Dataset with Natural and Diverse Instructions](https://mlsanity.com/paper/2601.03707/) (2026)
- [Generative AI on the Edge: Architecture and Performance Evaluation](https://mlsanity.com/paper/2411.17712/) (2024)
- [DVGBench: Implicit-to-Explicit Visual Grounding Benchmark in UAV Imagery with Large Vision-Language Models](https://mlsanity.com/paper/2601.00998/) (2026)
- [An LLM-based Self-Evolving Security Framework for 6G Space-Air-Ground Integrated Networks](https://mlsanity.com/paper/2505.03161/) (2025)
- [Efficient Onboard Vision-Language Inference in UAV-Enabled Low-Altitude Economy Networks via LLM-Enhanced Optimization](https://mlsanity.com/paper/2510.10028/) (2025)

## Built on by (most cited first, 2 of 2)

- [6GAgentGym: Tool Use, Data Synthesis, and Agentic Learning for Network Management](https://mlsanity.com/paper/2603.29656/) (2026)
- [6G Needs Agents: Toward Agentic AI-Native Networks for Autonomous Intelligence](https://mlsanity.com/paper/2605.01546/) (2026)

## Abstract (arXiv)

Autonomous unmanned aerial vehicle (UAV) systems are increasingly deployed in safety-critical, networked environments where they must operate reliably in the presence of malicious adversaries. While recent benchmarks have evaluated large language model (LLM)-based UAV agents in reasoning, navigation, and efficiency, systematic assessment of security, resilience, and trust under adversarial conditions remains largely unexplored, particularly in emerging 6G-enabled settings. We introduce α3-SecBench, the first large-scale evaluation suite for assessing the security-aware autonomy of LLM-based UAV agents under realistic adversarial interference. Building on multi-turn conversational UAV missions from α3-Bench, the framework augments benign episodes with 20,000 validated security overlay attack scenarios targeting seven autonomy layers, including sensing, perception, planning, control, communication, edge/cloud infrastructure, and LLM reasoning. α3-SecBench evaluates agents across three orthogonal dimensions: security (attack detection and vulnerability attribution), resilience (safe degradation behavior), and trust (policy-compliant tool usage). We evaluate 23 state-of-the-art LLMs from major industrial providers and leading AI labs using thousands of adversarially augmented UAV episodes sampled from a corpus of 113,475 missions spanning 175 threat types. While many models reliably detect anomalous behavior, effective mitigation, vulnerability attribution, and trustworthy control actions remain inconsistent. Normalized overall scores range from 12.9% to 57.1%, highlighting a significant gap between anomaly detection and security-aware autonomous decision-making. We release α3-SecBench on GitHub: https://github.com/maferrag/AlphaSecBench