# xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models

arXiv: https://arxiv.org/abs/2509.13021 · HTML: https://mlsanity.com/paper/2509.13021/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 19 of 19)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent](https://mlsanity.com/paper/2507.02259/) (2025)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities](https://mlsanity.com/paper/2503.17332/) (2025)
- [Teams of LLM Agents can Exploit Zero-Day Vulnerabilities](https://mlsanity.com/paper/2406.01637/) (2024)
- [NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Security](https://mlsanity.com/paper/2406.05590/) (2024)
- [PentestAgent: Incorporating LLM Agents to Automated Penetration Testing](https://mlsanity.com/paper/2411.05185/) (2024)
- [HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing](https://mlsanity.com/paper/2412.01778/) (2024)
- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [Deep hierarchical reinforcement agents for automated penetration testing](https://mlsanity.com/paper/2109.06449/) (2021)
- [Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements](https://mlsanity.com/paper/2410.17141/) (2024)
- [RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents](https://mlsanity.com/paper/2502.16730/) (2025)
- [AutoPentest: Enhancing Vulnerability Management With Autonomous LLM Agents](https://mlsanity.com/paper/2505.10321/) (2025)
- [RefPentester: A Knowledge-Informed Self-Reflective Penetration Testing Framework Based on Large Language Models](https://mlsanity.com/paper/2505.07089/) (2025)
- [Raij=u: Reinforcement Learning-Guided Post-Exploitation for Automating Security Assessment of Network Systems](https://mlsanity.com/paper/2309.15518/) (2023)

## Built on by (most cited first, 3 of 3)

- [What Makes a Good LLM Agent for Real-world Penetration Testing?](https://mlsanity.com/paper/2602.17622/) (2026)
- [Hackers or Hallucinators? A Comprehensive Analysis of LLM-Based Automated Penetration Testing](https://mlsanity.com/paper/2604.05719/) (2026)
- [The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents](https://mlsanity.com/paper/2608.06663/) (2026)

## Abstract (arXiv)

This work introduces xOffense, an AI-driven, multi-agent penetration testing framework that shifts the process from labor-intensive, expert-driven manual efforts to fully automated, machine-executable workflows capable of scaling seamlessly with computational infrastructure. At its core, xOffense leverages a fine-tuned, mid-scale open-source LLM (Qwen3-32B) to drive reasoning and decision-making in penetration testing. The framework assigns specialized agents to reconnaissance, vulnerability scanning, and exploitation, with an orchestration layer ensuring seamless coordination across phases. Fine-tuning on Chain-of-Thought penetration testing data further enables the model to generate precise tool commands and perform consistent multi-step reasoning. We evaluate xOffense on two rigorous benchmarks: AutoPenBench and AI-Pentest-Benchmark. The results demonstrate that xOffense consistently outperforms contemporary methods, achieving a sub-task completion rate of 79.17%, decisively surpassing leading systems such as VulnBot and PentestGPT. These findings highlight the potential of domain-adapted mid-scale LLMs, when embedded within structured multi-agent orchestration, to deliver superior, cost-efficient, and reproducible solutions for autonomous penetration testing.