mlsanityData to 7 Oct 2026

LLM Agents can Autonomously Hack Websites

arXiv 2402.06664 ↗First posted 6 Feb 2024cs.CR cs.AIcited by 108

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

In recent years, large language models (LLMs) have become increasingly capable and can now interact with tools (i.e., call functions), read documents, and recursively call themselves. As a result, these LLMs can now function autonomously as agents. With the rise in capabilities of these agents, recent work has speculated on how LLM agents would affect cybersecurity. However, not much is known about the offensive capabilities of LLM agents. In this work, we show that LLM agents can autonomously hack websites, performing tasks as complex as blind database schema extraction and SQL injections without human feedback. Importantly, the agent does not need to know the vulnerability beforehand. This capability is uniquely enabled by frontier models that are highly capable of tool use and leveraging extended context. Namely, we show that GPT-4 is capable of such hacks, but existing open-source models are not. Finally, we show that GPT-4 is capable of autonomously finding vulnerabilities in websites in the wild. Our findings raise questions about the widespread deployment of LLMs.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 23 of its 45 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

23 of 45 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  9. 2023Mistral 7Bcited by 3,065
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2022Emergent Abilities of Large Language Modelscited by 2,483
  12. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
11 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  3. 2023Augmented Language Models: a Surveycited by 398
  4. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  5. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  6. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  7. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  8. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  9. 2023Emergent autonomous scientific research capabilities of large language modelscited by 127
  10. 2020Text Data Augmentation: Towards better detection of spear-phishing emailscited by 9
  11. 2018Generative Models for Spear Phishing Posts on Social Mediacited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 40; 2026: 48.

108 papers.

Sort:

By area:

  1. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  2. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilitiescited by 93
  5. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  6. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilitiescited by 90
  7. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agentscited by 79
  8. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  9. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  10. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  11. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testingcited by 35
  12. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
38 more
  1. 2024Adversaries Can Misuse Combinations of Safe Modelscited by 31
  2. 2025LLM Agents Should Employ Security Principlescited by 28
  3. 2026The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systemscited by 27
  4. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  5. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  6. 2024When LLMs Go Online: The Emerging Threat of Web-Enabled LLMscited by 22
  7. 2025Cyber-Zero: Training Cybersecurity Agents without Runtimecited by 22
  8. 2025Training Language Model Agents to Find Vulnerabilities with CTF-Dojocited by 15
  9. 2026AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystemscited by 15
  10. 2025The AI Agent Indexcited by 14
  11. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 14
  12. 2025Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testingcited by 14
  13. 2024Hierarchical Context Pruning: Optimizing Real-World Code Completion with Repository-Level Pretrained Code LLMscited by 13
  14. 2026Measuring AI Agents' Progress on Multi-Step Cyber Attack Scenarioscited by 13
  15. 2024Catastrophic Cyber Capabilities Benchmark (3CB): Robustly Evaluating LLM Agent Cyber Offense Capabilitiescited by 12
  16. 2025OCCULT: Evaluating Large Language Models for Offensive Cyber Operation Capabilitiescited by 12
  17. 2025Prompt Injection 2.0: Hybrid AI Threatscited by 12
  18. 2024Can LLMs Understand Computer Networks? Towards a Virtual System Administratorcited by 11
  19. 2024Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvementscited by 11
  20. 2026Synthesizing Multi-Agent Harnesses for Vulnerability Discoverycited by 11
  21. 2025Jailbreak Attack Initializations as Extractors of Compliance Directionscited by 10
  22. 2025Automated Profile Inference with Language Model Agentscited by 9
  23. 2025A Frontier AI Risk Management Framework: Bridging the Gap Between Current AI Practices and Established Risk Managementcited by 8
  24. 2025Dynamic Risk Assessments for Offensive Cybersecurity Agentscited by 7
  25. 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 7
  26. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
  27. 2024CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical Researchercited by 6
  28. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 6
  29. 2025RedTeamLLM: an Agentic AI framework for offensive securitycited by 6
  30. 2025Concept-Level Explainability for Auditing & Steering LLM Responsescited by 6
  31. 2025Beyond Data Privacy: New Privacy Risks for Large Language Modelscited by 6
  32. 2026Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agentscited by 6
  33. 2024Applying Refusal-Vector Ablation to Llama 3.1 70B Agentscited by 5
  34. 2024Voice-Enabled AI Agents can Perform Common Scamscited by 5
  35. 2025LLMs unlock new paths to monetizing exploitscited by 5
  36. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  37. 2024What AI evaluations for preventing catastrophic risks can and cannot docited by 4
  38. 2025DefenderBench: A Toolkit for Evaluating Language Agents in Cybersecurity Environmentscited by 4

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities
  2. 2026From Prompt Injection to Web Exploitation: Revisiting Classic Vulnerabilities in LLM-Integrated Applications
  3. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
  4. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
  5. 2026LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity
  6. 2026ZeroDayBench: Evaluating LLM Agents on Unseen Zero-Day Vulnerabilities for Cyberdefense
  7. 2024LLM Agent Honeypot: Monitoring AI Hacking Agents in the Wild
  8. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models