# Towards Optimal Agentic Architectures for Offensive Security Tasks

arXiv: https://arxiv.org/abs/2604.18718 · HTML: https://mlsanity.com/paper/2604.18718/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 14 of 14)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback](https://mlsanity.com/paper/2306.14898/) (2023)
- [Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models](https://mlsanity.com/paper/2408.08926/) (2024)
- [Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models](https://mlsanity.com/paper/2312.04724/) (2023)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [Teams of LLM Agents can Exploit Zero-Day Vulnerabilities](https://mlsanity.com/paper/2406.01637/) (2024)
- [CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models](https://mlsanity.com/paper/2404.13161/) (2024)
- [Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets](https://mlsanity.com/paper/2604.02460/) (2026)
- [Single-agent or Multi-agent Systems? Why Not Both?](https://mlsanity.com/paper/2505.18286/) (2025)
- [Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline](https://mlsanity.com/paper/2601.12307/) (2026)
- [Multi-Agent Penetration Testing AI for the Web](https://mlsanity.com/paper/2508.20816/) (2025)
- [AI Agent Smart Contract Exploit Generation](https://mlsanity.com/paper/2507.05558/) (2025)
- [EVMbench: Evaluating AI Agents on Smart Contract Security](https://mlsanity.com/paper/2603.04915/) (2026)

## Built on by (most cited first, 4 of 4)

- [CrackMeBench: Binary Reverse Engineering for Agents](https://mlsanity.com/paper/2605.10597/) (2026)
- [Patch2Vuln: Agentic Reconstruction of Vulnerabilities from Linux Distribution Binary Patches](https://mlsanity.com/paper/2605.06601/) (2026)
- [Benchmarking Mythos-Linked Bug Rediscovery](https://mlsanity.com/paper/2605.17416/) (2026)
- [Transferable Self-Evolving Playbooks for Agentic Security Auditing](https://mlsanity.com/paper/2606.16420/) (2026)

## Abstract (arXiv)

Agentic security systems increasingly audit live targets with tool-using LLMs, but prior systems fix a single coordination topology, leaving unclear when additional agents help and when they only add cost. We treat topology choice as an empirical systems question. We introduce a controlled benchmark of 20 interactive targets (10 web/API and 10 binary), each exposing one endpoint-reachable ground-truth vulnerability, evaluated in whitebox and blackbox modes. The core study executes 600 runs over five architecture families, three model families, and both access modes, with a separate 60-run long-context pilot reported only in the appendix. On the completed core benchmark, detection-any reaches 58.0% and validated detection reaches 49.8%. MAS-Indep attains the highest validated detection rate (64.2%), while SAS is the strongest efficiency baseline at 0.058 per validated finding. Whitebox materially outperforms blackbox (67.0% vs. 32.7% validated detection), and web materially outperforms binary (74.3% vs. 25.3%). Bootstrap confidence intervals and paired target-level deltas show that the dominant effects are observability and domain, while some leading whitebox topologies remain statistically close. The main result is a non-monotonic cost-quality frontier: broader coordination can improve coverage, but it does not dominate once latency, token cost, and exploit-validation difficulty are taken into account.