mlsanityData to 14 Oct 2026

Red-Teaming the Agentic Red-Team

arXiv 2606.24496 ↗First posted 23 Jun 2026cs.CR cs.AIcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The use of agentic systems to perform offensive security operations has moved from a theoretical possibility to a commoditized capability. However, while the community has focused on creating more and more capable agents, less attention has been allocated to assessing the security of those systems. In this work, we present the first in-depth security analysis of the most widely used agentic systems for offensive security operations. We show that most of these tools share common design flaws that enable an active adversary to exfiltrate API keys, establish persistent footholds, and fully compromise the operator's machine, even when the agent operates inside a sandboxed container. To support our analysis, we introduce a full cyber kill chain for such agentic systems, capturing the progression from initial LLM manipulation to lateral movement, persistence, guardrail bypass, and sandbox escape. Building on our security analysis, we derive a robust architecture for agentic offensive-security tools and propose actionable, broadly applicable design principles that mitigate the disclosed attack paths at the architectural level.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 22 of its 46 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

22 of 46 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?11 May
  2. 2026Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoning10 May
  3. 2025Takedown: How It's Done in Modern Coding Agent Exploits29 Sep
  4. 2025BountyBench: Dollar Impact of AI Agent Attackers and Defenders on Real-World Cybersecurity Systems21 May
  5. 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction5 Mar
  6. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilities2 Jun

Other references · 16

  1. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  2. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 96
  3. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
  4. 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 83
  5. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  6. 2024An Empirical Evaluation of LLMs for Solving Offensive Security Challengescited by 35
  7. 2025Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networkscited by 17
  8. 2024Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattackscited by 15
  9. 2025Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testingcited by 15
  10. 2018Deep Reinforcement Fuzzingcited by 14
  11. 2019Improving Password Guessing via Representation Learningcited by 14
  12. 2026Synthesizing Multi-Agent Harnesses for Vulnerability Discoverycited by 13
4 more
  1. 2025Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerouscited by 12
  2. 2026AI Agents May Always Fall for Prompt Injectionscited by 9
  3. 2026CyberGym-E2E: Scalable Real-World Benchmark for AI Agents' End-to-End Cybersecurity Capabilitiescited by 8
  4. 2023Universal Neural-Cracking-Machines: Self-Configurable Password Models from Auxiliary Datacited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Mind the Gap: Comparing Model- vs Agentic-Level Red Teaming with Action-Graph Observability on GPT-OSS-20B
  2. 2025A Survey on Agentic Security: Applications, Threats and Defenses
  3. 2026MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring
  4. 2026AgenticRed: Evolving Agentic Systems for Red-Teaming
  5. 2026Autonomous Adversary: Red-Teaming in the age of LLM
  6. 2026Securing the AI Agent: A Unified Framework for Multi-Layer Agent Red Teaming
  7. 2026Redefining AI Red Teaming in the Agentic Era: From Weeks to Hours
  8. 2026Governing Execution Risk in Agentic AI Systems: A Trajectory-Guided Framework for Red Teaming