# The Ethics of Autonomous AI Agents for Offensive Security

arXiv: https://arxiv.org/abs/2607.20255 · HTML: https://mlsanity.com/paper/2607.20255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 67 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 17 of 17)

- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://mlsanity.com/paper/1802.07228/) (2018)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [Large Language Models Reflect the Ideology of their Creators](https://mlsanity.com/paper/2410.18417/) (2024)
- [Structured access: an emerging paradigm for safe AI deployment](https://mlsanity.com/paper/2201.05159/) (2022)
- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks](https://mlsanity.com/paper/2310.11409/) (2023)
- [Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks](https://mlsanity.com/paper/2501.16466/) (2025)
- [Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks](https://mlsanity.com/paper/2502.04227/) (2025)
- [Understanding Hackers' Work: An Empirical Study of Offensive Security Practitioners](https://mlsanity.com/paper/2308.07057/) (2023)
- [AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?](https://mlsanity.com/paper/2411.01236/) (2024)
- [Adversarial Bug Reports as a Security Risk in Language Model-Based Automated Program Repair](https://mlsanity.com/paper/2509.05372/) (2025)
- [CAI Fluency: A Framework for Cybersecurity AI Fluency](https://mlsanity.com/paper/2508.13588/) (2025)
- [Cochise: A Reference Harness for Autonomous Penetration Testing](https://mlsanity.com/paper/2605.11671/) (2026)
- [Updating the Wassenaar Debate Once Again: Surveillance, Intrusion Software, and Ambiguity](https://mlsanity.com/paper/1906.02235/) (2019)
- [Criminal Liability of Generative Artificial Intelligence Providers for User-Generated Child Sexual Abuse Material](https://mlsanity.com/paper/2601.03788/) (2026)
- [Enhancing Linux Privilege Escalation Attack Capabilities of Local LLM Agents](https://mlsanity.com/paper/2604.27143/) (2026)

## Built on by (most cited first, 1 of 1)

- [Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response](https://mlsanity.com/paper/2607.25379/) (2026)

## Abstract (arXiv)

LLM-driven autonomous agents are reshaping offensive security. Unlike traditional penetration-testing tooling - deterministic, narrowly scoped, and operated by trained practitioners - agentic security tools exhibit indeterminacy along three independent dimensions. First, their actions are drawn from a non-deterministic policy whose outputs resist both ex-ante and ex-post explanation. This complicates incident attribution and pre-deployment safety reviews. Second, their impact is open-ended due to their non-deterministic actions, agency of utilized models, and opaque LLM supply-chains. Third, their user population is indeterminate in both size and required skill: the operating skill floor for using or developing offensive capabilities has dropped sharply. These three properties are linked thematically, but are not derivable from one another. Combined with the structural cost asymmetry between offense and defense, they enable the industrialization of offensive capability. The net short-term effect favors attackers, even if the same technology may, in the long run, democratize access to defensive practice. Existing dual-use cybersecurity and AI-ethics frameworks struggle to address this combination. Our work analyzes how moral attribution becomes diffuse between users, tool-makers, and third parties when employing autonomous AI agents for offensive security. We also examine the stakeholder impact of this technology and provide stratified recommendations.