mlsanityData to 7 Oct 2026

Teams of LLM Agents can Exploit Zero-Day Vulnerabilities

arXiv 2406.01637 ↗First posted 2 Jun 2024cs.MA cs.AIcited by 93

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM agents have become increasingly sophisticated, especially in the realm of cybersecurity. Researchers have shown that LLM agents can exploit real-world vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. However, these agents still perform poorly on real-world vulnerabilities that are unknown to the agent ahead of time (zero-day vulnerabilities). In this work, we show that teams of LLM agents can exploit real-world, zero-day vulnerabilities. Prior agents struggle with exploring many different vulnerabilities and long-range planning when used alone. To resolve this, we introduce HPTSA, a system of agents with a planning agent that can launch subagents. The planning agent explores the system and determines which subagents to call, resolving long-term planning issues when trying different vulnerabilities. We construct a benchmark of 14 real-world vulnerabilities and show that our team of agents improve over prior agent frameworks by up to 4.3X.

Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 29 of its 70 references that are in the map. First posted on arXiv 2 Jun 2024.

Builds on

29 of 70 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr
  2. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  3. 2024LLM Agents can Autonomously Hack Websites6 Feb

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
14 more
  1. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  2. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  3. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  4. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 231
  5. 2023AutoAgents: A Framework for Automatic Agent Generationcited by 211
  6. 2022TALM: Tool Augmented Language Modelscited by 197
  7. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  8. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  9. 2023Chain of Hindsight Aligns Language Models with Feedbackcited by 147
  10. 2024MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolutioncited by 132
  11. 2023Getting pwn'd by AI: Penetration Testing with Large Language Modelscited by 82
  12. 2024MACRec: a Multi-Agent Collaboration Framework for Recommendationcited by 58
  13. 2024Holistic Safety and Responsibility Evaluations of Advanced AI Modelscited by 18
  14. 2022Will AI Make Cyber Swords or Shields: A few mathematical models of technological progresscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 32; 2026: 48.

93 papers.

Sort:

By area:

  1. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 124
  2. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilitiescited by 90
  3. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  4. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  5. 2025VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Frameworkcited by 30
  6. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 28
  7. 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agentscited by 28
  8. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  9. 2024AutoPenBench: Benchmarking Generative Agents for Penetration Testingcited by 20
  10. 2026SkillSieve: A Hierarchical Triage Framework for Detecting Malicious AI Agent Skillscited by 18
  11. 2025Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networkscited by 17
  12. 2025Estimating Worst-Case Frontier Risks of Open-Weight LLMscited by 17
38 more
  1. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 17
  2. 2026Temporary Authority, Permanent Effects: Commit-Time Authorization for LLM Agentscited by 17
  3. 2024Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattackscited by 15
  4. 2025The AI Agent Indexcited by 14
  5. 2025Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challengescited by 14
  6. 2025Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testingcited by 14
  7. 2025Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Designcited by 13
  8. 2026Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarioscited by 13
  9. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  10. 2024Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvementscited by 11
  11. 2025FaultLine: Automated Proof-of-Vulnerability Generation Using LLM Agentscited by 11
  12. 2026Synthesizing Multi-Agent Harnesses for Vulnerability Discoverycited by 11
  13. 2026A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Frameworkcited by 9
  14. 2025AutoPentest: Enhancing Vulnerability Management With Autonomous LLM Agentscited by 8
  15. 2026Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidancecited by 8
  16. 2025Dynamic Risk Assessments for Offensive Cybersecurity Agentscited by 7
  17. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
  18. 2024The Role of Governments in Increasing Interconnected Post-Deployment Monitoring of AIcited by 6
  19. 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
  20. 2026Bounded Agents: Delegation Security for Multi-Agent AI Systemscited by 6
  21. 2024Applying Refusal-Vector Ablation to Llama 3.1 70B Agentscited by 5
  22. 2024Voice-Enabled AI Agents can Perform Common Scamscited by 5
  23. 2025LLMs unlock new paths to monetizing exploitscited by 5
  24. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  25. 2024Declare and Justify: Explicit assumptions in AI evaluations are necessary for effective regulationcited by 4
  26. 2024What AI evaluations for preventing catastrophic risks can and cannot docited by 4
  27. 2025Improving LLM Agents with Reinforcement Learning on Cryptographic CTF Challengescited by 4
  28. 2025Agentic Discovery and Validation of Android App Vulnerabilitiescited by 4
  29. 2025LLM Agents for Automated Web Vulnerability Reproduction: Are We There Yet?cited by 4
  30. 2026Towards Optimal Agentic Architectures for Offensive Security Taskscited by 4
  31. 2026AgentCyberRange: Benchmarking Frontier AI Systems in Realistic Cyber Rangescited by 4
  32. 2024Unelicitable Backdoors in Language Models via Cryptographic Transformer Circuitscited by 3
  33. 2025Love, Lies, and Language Models: Investigating AI's Role in Romance-Baiting Scamscited by 3
  34. 2026Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generationcited by 3
  35. 2026ROGUE: Evaluating Corrigibility Failures in Frontier Computer-Use Agentscited by 3
  36. 2025Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunitiescited by 2
  37. 2025xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Modelscited by 2
  38. 2025When "Correct" Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents?cited by 2

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense
  2. 2026ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?
  3. 2026PlanFlip: Attacking Multi-Agent LLM Systems via Planning-Phase Prompt Injection
  4. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks
  5. 2025Towards Effective Offensive Security LLM Agents: Hyperparameter Tuning, LLM as a Judge, and a Lightweight CTF Benchmark
  6. 2026Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities
  7. 2026Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming
  8. 2026LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity