mlsanityData to 7 Oct 2026

xOffense: An Autonomous Multi-Agent Framework for Penetration Testing with Domain-Adapted Large Language Models

arXiv 2509.13021 ↗First posted 16 Sep 2025cs.CR cs.AIcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

This work introduces xOffense, an AI-driven, multi-agent penetration testing framework that shifts the process from labor-intensive, expert-driven manual efforts to fully automated, machine-executable workflows capable of scaling seamlessly with computational infrastructure. At its core, xOffense leverages a fine-tuned, mid-scale open-source LLM (Qwen3-32B) to drive reasoning and decision-making in penetration testing. The framework assigns specialized agents to reconnaissance, vulnerability scanning, and exploitation, with an orchestration layer ensuring seamless coordination across phases. Fine-tuning on Chain-of-Thought penetration testing data further enables the model to generate precise tool commands and perform consistent multi-step reasoning. We evaluate xOffense on two rigorous benchmarks: AutoPenBench and AI-Pentest-Benchmark. The results demonstrate that xOffense consistently outperforms contemporary methods, achieving a sub-task completion rate of 79.17%, decisively surpassing leading systems such as VulnBot and PentestGPT. These findings highlight the potential of domain-adapted mid-scale LLMs, when embedded within structured multi-agent orchestration, to deliver superior, cost-efficient, and reproducible solutions for autonomous penetration testing.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 19 of its 41 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

19 of 41 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025AutoPentest: Enhancing Vulnerability Management With Autonomous LLM Agents15 May
  2. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities21 Mar
  3. 2025RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents23 Feb
  4. 2025VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework23 Jan
  5. 2024HackSynth: LLM Agent and Evaluation Framework for Autonomous Penetration Testing2 Dec
  6. 2024PentestAgent: Incorporating LLM Agents to Automated Penetration Testing7 Nov
  7. 2024Teams of LLM Agents can Exploit Zero-Day Vulnerabilities2 Jun
  8. 2024LLM Agents can Autonomously Exploit One-day Vulnerabilities11 Apr

Other references · 11

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  3. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  4. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,278
  5. 2025MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agentcited by 235
  6. 2024AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attackscited by 94
  7. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 90
  8. 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
  9. 2024Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvementscited by 11
  10. 2025RefPentester: A Knowledge-Informed Self-Reflective Penetration Testing Framework Based on Large Language Modelscited by 2
  11. 2023Raij=u: Reinforcement Learning-Guided Post-Exploitation for Automating Security Assessment of Network Systemscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026What Makes a Good LLM Agent for Real-world Penetration Testing?cited by 8
  2. 2026The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing
  2. 2026ZERO-APT: A Closed-Loop Adversarial Framework for LLM-Driven Automated Penetration Testing under Intelligent Defense ↗
  3. 2026Baselines Before Architecture: Evaluating Coding Agents for Autonomous Penetration Testing
  4. 2024AutoPenBench: Benchmarking Generative Agents for Penetration Testing
  5. 2026APT-Agent: Automated Penetration Testing using Large Language Models
  6. 2025Guided Reasoning in LLM-Driven Penetration Testing Using Structured Attack Trees
  7. 2025Automated Penetration Testing with LLM Agents and Classical Planning
  8. 2026Big Enough to Break Out: Tracking the Rising Capability of LLM Penetration-Testing Agents