# Hacking, The Lazy Way: LLM Augmented Pentesting

arXiv: https://arxiv.org/abs/2409.09493 · HTML: https://mlsanity.com/paper/2409.09493/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 14 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [Examining Zero-Shot Vulnerability Repair with Large Language Models](https://mlsanity.com/paper/2112.02125/) (2021)
- [PentestGPT: An LLM-empowered Automatic Penetration Testing Tool](https://mlsanity.com/paper/2308.06782/) (2023)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [Using Large Language Models for Cybersecurity Capture-The-Flag Challenges and Certification Questions](https://mlsanity.com/paper/2308.10443/) (2023)
- [CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical Researcher](https://mlsanity.com/paper/2408.11650/) (2024)

## Built on by (most cited first, 5 of 5)

- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks](https://mlsanity.com/paper/2410.20911/) (2024)
- [A Contemporary Survey of Large Language Model Assisted Program Analysis](https://mlsanity.com/paper/2502.18474/) (2025)
- [Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research](https://mlsanity.com/paper/2506.08693/) (2025)
- [Environment-Grounded Multi-Agent Workflow for Autonomous Penetration Testing](https://mlsanity.com/paper/2603.24221/) (2026)

## Abstract (arXiv)

In our research, we introduce a new concept called "LLM Augmented Pentesting" demonstrated with a tool named "Pentest Copilot," that revolutionizes the field of ethical hacking by integrating Large Language Models (LLMs) into penetration testing workflows, leveraging the advanced GPT-4-turbo model. Our approach focuses on overcoming the traditional resistance to automation in penetration testing by employing LLMs to automate specific sub-tasks while ensuring a comprehensive understanding of the overall testing process. Pentest Copilot showcases remarkable proficiency in tasks such as utilizing testing tools, interpreting outputs, and suggesting follow-up actions, efficiently bridging the gap between automated systems and human expertise. By integrating a "chain of thought" mechanism, Pentest Copilot optimizes token usage and enhances decision-making processes, leading to more accurate and context-aware outputs. Additionally, our implementation of Retrieval-Augmented Generation (RAG) minimizes hallucinations and ensures the tool remains aligned with the latest cybersecurity techniques and knowledge. We also highlight a unique infrastructure system that supports in-browser penetration testing, providing a robust platform for cybersecurity professionals. Our findings demonstrate that LLM Augmented Pentesting can not only significantly enhance task completion rates in penetration testing but also effectively addresses real-world challenges, marking a substantial advancement in the cybersecurity domain.