# RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents

arXiv: https://arxiv.org/abs/2502.16730 · HTML: https://mlsanity.com/paper/2502.16730/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation](https://mlsanity.com/paper/2407.17788/) (2024)
- [A Unified Modeling Framework for Automated Penetration Testing](https://mlsanity.com/paper/2502.11588/) (2025)

## Built on by (most cited first, 11 of 11)

- [Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design](https://mlsanity.com/paper/2504.10112/) (2025)
- [What Makes a Good LLM Agent for Real-world Penetration Testing?](https://mlsanity.com/paper/2602.17622/) (2026)
- [PACEbench: A Framework for Evaluating Practical AI Cyber-Exploitation Capabilities](https://mlsanity.com/paper/2510.11688/) (2025)
- [Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing](https://mlsanity.com/paper/2604.05719/) (2026)
- [xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models](https://mlsanity.com/paper/2509.13021/) (2025)
- [Living Off the LLM: How LLMs Will Change Adversary Tactics](https://mlsanity.com/paper/2510.11398/) (2025)
- [Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFs](https://mlsanity.com/paper/2510.17521/) (2025)
- [CritBench: A Framework for Evaluating Cybersecurity Capabilities of Large Language Models in IEC 61850 Digital Substation Environments](https://mlsanity.com/paper/2604.06019/) (2026)
- [Recognition Without Mitigation: Ethical Frameworks in Autonomous Offensive-LLM Agent Research](https://mlsanity.com/paper/2506.08693/) (2025)
- [SoK: Measuring What Matters for Closed-Loop Security Agents](https://mlsanity.com/paper/2510.01654/) (2025)
- [From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World](https://mlsanity.com/paper/2605.10834/) (2026)

## Abstract (arXiv)

We present RapidPen, a fully automated penetration testing (pentesting) framework that addresses the challenge of achieving an initial foothold (IP-to-Shell) without human intervention. Unlike prior approaches that focus primarily on post-exploitation or require a human-in-the-loop, RapidPen leverages large language models (LLMs) to autonomously discover and exploit vulnerabilities, starting from a single IP address. By integrating advanced ReAct-style task planning (Re) with retrieval-augmented knowledge bases of successful exploits, along with a command-generation and direct execution feedback loop (Act), RapidPen systematically scans services, identifies viable attack vectors, and executes targeted exploits in a fully automated manner. In our evaluation against a vulnerable target from the Hack The Box platform, RapidPen achieved shell access within 200-400 seconds at a per-run cost of approximately 0.3-0.6, demonstrating a 60% success rate when reusing prior "success-case" data. These results underscore the potential of truly autonomous pentesting for both security novices and seasoned professionals. Organizations without dedicated security teams can leverage RapidPen to quickly identify critical vulnerabilities, while expert pentesters can offload repetitive tasks and focus on complex challenges. Ultimately, our work aims to make penetration testing more accessible and cost-efficient, thereby enhancing the overall security posture of modern software ecosystems. Fore more information, visit this link: https://secdevlab.com/rapidpen