mlsanityData to 14 Oct 2026

Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response

arXiv 2607.25379 ↗First posted 28 Jul 2026cs.AIcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Cyber-capable AI agents combine language models with tools, memory, and execution environments to perform multi-step offensive-security tasks. Existing work separately measures cyber capability and catalogs attacks against agent components, but provides less guidance on containing a capable agent within the environments used to evaluate it. This review synthesizes five vulnerability classes at that boundary: multi-step offensive chains, objectives that conflict with sandbox boundaries, supply-chain and credential exposure, persistent command-and-control, and the speed of automated action. We use two separate preliminary incident records: the reported July 2026 Hugging Face/OpenAI evaluation breach and Anthropic's subsequent three-incident evaluation review. A comparative evidence protocol distinguishes record-specific factual claims from the shared systems lesson: the evaluation environment is itself part of the security boundary. Across the taxonomy and records, we examine controls for containment, privilege separation, provenance, and responder access, including the dual-use problem that defensive artifacts may also enable misuse. The review identifies practical priorities for evaluating cyber capability together with the security of the environment in which that capability is exercised.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 85 of its 93 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

85 of 93 references.

Closest ancestors · 35

Same direction, newest first

  1. 2026The Ethics of Autonomous AI Agents for Offensive Security22 Jul
  2. 2026When Agents Remember Too Much: Memory Poisoning Attacks on Large Language Model Agents6 Jul
  3. 2026LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity26 Jun
  4. 2026When Lower Privileges Suffice: Investigating Over-Privileged Tool Selection in LLM Agents18 Jun
  5. 2026A New Framework for Cybersecurity Refusals in AI Agents31 May
  6. 2026APT-Agent: Automated Penetration Testing using Large Language Models24 May
  7. 2026Detecting Offensive Cyber Agents: A Detection-in-Depth Approach21 May
  8. 2026CTFusion: A CTF-based Benchmark for LLM Agent Evaluation12 May
  9. 2026ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?11 May
  10. 2026Autonomous LLM Agents & CTFs: A Second Look29 Apr
  11. 2026When the Agent Is the Adversary: Architectural Requirements for Agentic AI Containment After the April 2026 Frontier Model Escape25 Apr
  12. 2026Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges21 Apr
23 more
  1. 2026Supply-Chain Poisoning Attacks Against LLM Coding Agent Skill Ecosystems3 Apr
  2. 2026Caging the Agents: A Zero Trust Security Architecture for Autonomous AI in Healthcare18 Mar
  3. 2026CTFExplorer: Evaluating LLM Offensive Agents Through Multi-Target Web CTF Benchmarking8 Feb
  4. 2026Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations5 Feb
  5. 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agents9 Jan
  6. 2025Detecting Sleeper Agents in Large Language Models via Semantic Drift Analysis20 Nov
  7. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges27 Oct
  8. 2025CORTEX: Collaborative LLM Agents for High-Stakes Alert Triage30 Sep
  9. 2025Training Language Model Agents to Find Vulnerabilities with CTF-Dojo25 Aug
  10. 2025Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark5 Aug
  11. 2025SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks13 Jun
  12. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities21 Mar
  13. 2025D-CIPHER: Dynamic Collaborative Intelligent Multi-Agent System with Planner and Heterogeneous Executors for Offensive Security15 Feb
  14. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing2 Dec
  15. 2024LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild17 Oct
  16. 2024Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilities10 Oct
  17. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies28 Jul
  18. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilities2 Jun
  19. 2024Artificial Intelligence as the New Hacker: Developing Agents for Offensive Security9 May
  20. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr
  21. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  22. 2024LLM Agents can Autonomously Hack Websites6 Feb
  23. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan

Other references · 50

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  6. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  7. 2022Red Teaming Language Models with Language Modelscited by 1,026
  8. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  9. 2024Alignment faking in large language modelscited by 360
  10. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
  11. 2021Trustworthy AI: From Principles to Practicescited by 207
  12. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
38 more
  1. 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 164
  2. 2022Compute Trends Across Three Eras of Machine Learningcited by 163
  3. 2025Measuring AI Ability to Complete Long Software Taskscited by 156
  4. 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
  5. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  6. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 127
  7. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  8. 2019Optimal Policies Tend to Seek Powercited by 119
  9. 2023PentestGPT: An LLM-empowered Automatic Penetration Testing Toolcited by 97
  10. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 93
  11. 2024CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Modelscited by 89
  12. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  13. 2024CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Modelscited by 58
  14. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 50
  15. 2021Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-Level Backdoor Attackscited by 49
  16. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  17. 2023Power-seeking can be probable and predictive for trained agentscited by 25
  18. 2024SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecuritycited by 21
  19. 2024Models Are Codes: Towards Measuring Malicious Code Poisoning Attacks on Pre-trained Model Hubscited by 18
  20. 2025The Elicitation Game: Evaluating Capability Elicitation Techniquescited by 18
  21. 2026Quantifying Frontier LLM Capabilities for Container Sandbox Escapecited by 18
  22. 2024A Large-Scale Exploit Instrumentation Study of AI/ML Supply Chain Attacks in Hugging Face Modelscited by 14
  23. 2025LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centrescited by 14
  24. 2026Audit Trails for Accountability in Large Language Modelscited by 12
  25. 2026Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenderscited by 10
  26. 2024Benchmark Early and Red Team Often: A Framework for Assessing and Managing Dual-Use Hazards of AI Foundation Modelscited by 9
  27. 2024Model Supply Chain Poisoning: Backdooring Pre-trained Models via Embedding Indistinguishabilitycited by 6
  28. 2026Beyond Static Sandboxing: Learned Capability Governance for Autonomous AI Agentscited by 4
  29. 2025Mitigating Cyber Risk in the Age of Open-Weight LLMs: Policy Gaps and Technical Realitiescited by 3
  30. 2025PickleBall: Secure Deserialization of Pickle-based Machine Learning Models (Extended Report)cited by 3
  31. 2025Beyond Over-Refusal: Scenario-Based Diagnostics and Post-Hoc Mitigation for Exaggerated Refusals in LLMscited by 3
  32. 2025Cybersecurity AI: The World's Top AI Agent for Security Capture-the-Flag (CTF)cited by 2
  33. 2025Can Safety Fine-Tuning Be More Principled? Lessons Learned from Cybersecuritycited by 1
  34. 2026SafePickle: Robust and Generic ML Detection of Malicious Pickle-based ML Modelscited by 1
  35. 2026Mythos and the Unverified Cage: Z3-Based Pre-Deployment Verification for Frontier-Model Sandbox Infrastructurecited by 1
  36. 2026Toward Autonomous SOC Operations: End-to-End LLM Framework for Threat Detection, Query Generation, and Resolution in Security Operationscited by 1
  37. 2026Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applicationscited by 1
  38. 2026SysAdmin: Measuring Instrumental Power-Seeking in Frontier AIcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026From Training to Escalation: Forensic Analysis and Local Recreation of RL/RLHF-Induced Privilege Escalation in AI Agents (CVE-2026-65616)cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026To Defend Against Cyber Attacks, We Must Teach AI Agents to Hack
  2. 2025A Framework for Evaluating Emerging Cyberattack Capabilities of AI
  3. 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities
  4. 2025Safety by Measurement: A Systematic Literature Review of AI Safety Evaluation Methods
  5. 2026CWE-bench: Refocusing Agentic Security from Exploitation to Defense
  6. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
  7. 2026Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarios
  8. 2025Dynamic Risk Assessments for Offensive Cybersecurity Agents