# PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation

arXiv: https://arxiv.org/abs/2407.17788 · HTML: https://mlsanity.com/paper/2407.17788/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2024.

## Built on by (most cited first, 30 of 30)

- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [Multi-Agent Penetration Testing AI for the Web](https://mlsanity.com/paper/2508.20816/) (2025)
- [Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks](https://mlsanity.com/paper/2501.16466/) (2025)
- [A Framework for Evaluating Emerging Cyberattack Capabilities of AI](https://mlsanity.com/paper/2503.11917/) (2025)
- [Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges](https://mlsanity.com/paper/2506.17644/) (2025)
- [Benchmarking Practices in LLM-driven Offensive Security: Testbeds, Metrics, and Experiment Design](https://mlsanity.com/paper/2504.10112/) (2025)
- [OCCULT: Evaluating Large Language Models for Offensive Cyber Operation Capabilities](https://mlsanity.com/paper/2502.15797/) (2025)
- [RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents](https://mlsanity.com/paper/2502.16730/) (2025)
- [A Contemporary Survey of Large Language Model Assisted Program Analysis](https://mlsanity.com/paper/2502.18474/) (2025)
- [Dynamic Risk Assessments for Offensive Cybersecurity Agents](https://mlsanity.com/paper/2505.18384/) (2025)
- [PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts](https://mlsanity.com/paper/2511.02780/) (2025)
- [PentestEval: Benchmarking LLM-based Penetration Testing with Modular and Stage-Level Design](https://mlsanity.com/paper/2512.14233/) (2025)
- [AutoPentester: An LLM Agent-based Framework for Automated Pentesting](https://mlsanity.com/paper/2510.05605/) (2025)
- [ARACNE: An LLM-Based Autonomous Shell Pentesting Agent](https://mlsanity.com/paper/2502.18528/) (2025)
- [Automated Penetration Testing with LLM Agents and Classical Planning](https://mlsanity.com/paper/2512.11143/) (2025)
- [Agentic AI for Cyber Resilience: A New Security Paradigm and Its System-Theoretic Foundations](https://mlsanity.com/paper/2512.22883/) (2025)
- [The Best Defense is a Good Offense: Countering LLM-Powered Cyberattacks](https://mlsanity.com/paper/2410.15396/) (2024)
- [Detecting Malicious Source Code in PyPI Packages with LLMs: Does RAG Come in Handy?](https://mlsanity.com/paper/2504.13769/) (2025)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [HackWorld: Evaluating Computer-Use Agents on Exploiting Web Application Vulnerabilities](https://mlsanity.com/paper/2510.12200/) (2025)
- [APT-Agent: Automated Penetration Testing using Large Language Models](https://mlsanity.com/paper/2605.24949/) (2026)
- [Construction and Evaluation of LLM-based agents for Semi-Autonomous penetration testing](https://mlsanity.com/paper/2502.15506/) (2025)
- [ACSE-Eval: Can LLMs threat model real-world cloud infrastructure?](https://mlsanity.com/paper/2505.11565/) (2025)
- [Cybersecurity AI: Evaluating Agentic Cybersecurity in Attack/Defense CTFs](https://mlsanity.com/paper/2510.17521/) (2025)
- [Enhancing Linux Privilege Escalation Attack Capabilities of Local LLM Agents](https://mlsanity.com/paper/2604.27143/) (2026)
- [The Game-Theoretic Symbiosis of Trust and AI in Networked Systems](https://mlsanity.com/paper/2411.12859/) (2024)
- [From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing](https://mlsanity.com/paper/2509.14289/) (2025)
- [AutoEG: Exploiting Known Third-Party Vulnerabilities in Black-Box Web Applications](https://mlsanity.com/paper/2604.00704/) (2026)
- [Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives](https://mlsanity.com/paper/2609.16694/) (2026)
- [PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs](https://mlsanity.com/paper/2609.18120/) (2026)

## Abstract (arXiv)

Recent advances in Large Language Models (LLMs) have shown significant potential in enhancing cybersecurity defenses against sophisticated threats. LLM-based penetration testing is an essential step in automating system security evaluations by identifying vulnerabilities. Remediation, the subsequent crucial step, addresses these discovered vulnerabilities. Since details about vulnerabilities, exploitation methods, and software versions offer crucial insights into system weaknesses, integrating penetration testing with vulnerability remediation into a cohesive system has become both intuitive and necessary. This paper introduces PenHeal, a two-stage LLM-based framework designed to autonomously identify and mitigate security vulnerabilities. The framework integrates two LLM-enabled components: the Pentest Module, which detects multiple vulnerabilities within a system, and the Remediation Module, which recommends optimal remediation strategies. The integration is facilitated through Counterfactual Prompting and an Instructor module that guides the LLMs using external knowledge to explore multiple potential attack paths effectively. Our experimental results demonstrate that PenHeal not only automates the identification and remediation of vulnerabilities but also significantly improves vulnerability coverage by 31%, increases the effectiveness of remediation strategies by 32%, and reduces the associated costs by 46% compared to baseline models. These outcomes highlight the transformative potential of LLMs in reshaping cybersecurity practices, offering an innovative solution to defend against cyber threats.