mlsanityData to 7 Oct 2026

Getting pwn'd by AI: Penetration Testing with Large Language Models

arXiv 2308.00121 ↗First posted 24 Jul 2023cs.CL cs.AI cs.CRcited by 82

Not on the map. Why?

Cited by 33 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The field of software security testing, more specifically penetration testing, is an activity that requires high levels of expertise and involves many manual testing and analysis steps. This paper explores the potential usage of large-language models, such as GPT3.5, to augment penetration testers with AI sparring partners. We explore the feasibility of supplementing penetration testers with AI models for two distinct use cases: high-level task planning for security testing assignments and low-level vulnerability hunting within a vulnerable virtual machine. For the latter, we implemented a closed-feedback loop between LLM-generated low-level actions with a vulnerable virtual machine (connected through SSH) and allowed the LLM to analyze the machine state for vulnerabilities and suggest concrete attack vectors which were automatically executed within the virtual machine. We discuss promising initial results, detail avenues for improvement, and close deliberating on the ethics of providing AI-based sparring partners.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). Builds on 11 of its 42 references that are in the map. First posted on arXiv 24 Jul 2023.

Builds on

11 of 42 references.

References · 11

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2022Emergent Abilities of Large Language Modelscited by 2,483
  3. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  4. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  5. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  6. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  7. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
  8. 2013Stabilitycited by 119
  9. 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 81
  10. 2022Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Modelscited by 70
  11. 2022The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligencecited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 29; 2026: 24.

82 papers.

Sort:

By area:

  1. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  2. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 380
  3. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilitiescited by 125
  4. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 94
  5. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilitiescited by 93
  6. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  7. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  8. 2024PentestAgent: Incorporating LLM Agents to Automated Penetration Testingcited by 46
  9. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testingcited by 35
  10. 2024Large Language Models in Cybersecurity: State-of-the-Artcited by 34
  11. 2025VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Frameworkcited by 30
  12. 2023LLMs as Hackers: Autonomous Linux Privilege Escalation Attackscited by 25
38 more
  1. 2023LLM for SoC Security: A Paradigm Shiftcited by 22
  2. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  3. 2024AutoPenBench: Benchmarking Generative Agents for Penetration Testingcited by 20
  4. 2023Out of the Cage: How Stochastic Parrots Win in Cyber Security Environmentscited by 17
  5. 2025Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networkscited by 17
  6. 2024Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattackscited by 15
  7. 2025AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defensescited by 14
  8. 2025Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challengescited by 14
  9. 2025Personalized Attacks of Social Engineering in Multi-turn Conversations: LLM Agents for Simulation and Detectioncited by 13
  10. 2025Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Designcited by 13
  11. 2024Just another copy and paste? Comparing the security vulnerabilities of ChatGPT generated code and StackOverflow answerscited by 12
  12. 2024Can LLMs Understand Computer Networks? Towards a Virtual System Administratorcited by 11
  13. 2025RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agentscited by 10
  14. 2023Cyber Sentinel: Exploring Conversational Agents in Streamlining Security Tasks with GPT-4cited by 9
  15. 2024Hackphyr: A Local Fine-Tuned LLM Agent for Network Security Environmentscited by 9
  16. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Securitycited by 9
  17. 2024A Survey of Large Language Models in Cybersecuritycited by 8
  18. 2024MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generationcited by 8
  19. 2025Do LLMs Consider Security? An Empirical Study on Responses to Programming Questionscited by 8
  20. 2025Dynamic Risk Assessments for Offensive Cybersecurity Agentscited by 7
  21. 2025PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contractscited by 7
  22. 2024A Jailbroken GenAI Model Can Cause Substantial Harm: GenAI-powered Applications are Vulnerable to PromptWarescited by 6
  23. 2024CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical Researchercited by 6
  24. 2025PentestEval: Benchmarking LLM-based Penetration Testing with Modular and Stage-Level Designcited by 6
  25. 2024Hacking, The Lazy Way: LLM Augmented Pentestingcited by 5
  26. 2024SoK: On the Offensive Potential of AIcited by 5
  27. 2025LLMs unlock new paths to monetizing exploitscited by 5
  28. 2025Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmapcited by 5
  29. 2023Survey on AI Ethics: A Socio-technical Perspectivecited by 4
  30. 2025Benchmarking LLM-Assisted Blue Teaming via Standardized Threat Huntingcited by 4
  31. 2026Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOpscited by 4
  32. 2026Cochise: A Reference Harness for Autonomous Penetration Testingcited by 3
  33. 2024A Preliminary Study on Using Large Language Models in Software Pentestingcited by 2
  34. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  35. 2025Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilotcited by 2
  36. 2025Towards AI-Driven Human-Machine Co-Teaming for Adaptive and Agile Cyber Security Operation Centerscited by 2
  37. 2025Navigating the growing field of research on AI for software testing -- the taxonomy for AI-augmented software testing and an ontology-driven literature surveycited by 2
  38. 2025Large Language Models for Software Testing: A Research Roadmapcited by 2

50 most cited of 82 All 82 (JSON)