# Cochise: A Reference Harness for Autonomous Penetration Testing

arXiv: https://arxiv.org/abs/2605.11671 · HTML: https://mlsanity.com/paper/2605.11671/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 6 of 6)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Getting pwn'd by AI: Penetration Testing with Large Language Models](https://mlsanity.com/paper/2308.00121/) (2023)
- [Incalmo: An Autonomous LLM-assisted System for Red Teaming Multi-Host Networks](https://mlsanity.com/paper/2501.16466/) (2025)
- [CAI: An Open, Bug Bounty-Ready Cybersecurity AI](https://mlsanity.com/paper/2504.06017/) (2025)
- [Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks](https://mlsanity.com/paper/2502.04227/) (2025)
- [What Makes a Good LLM Agent for Real-world Penetration Testing?](https://mlsanity.com/paper/2602.17622/) (2026)

## Built on by (most cited first, 3 of 3)

- [The Ethics of Autonomous AI Agents for Offensive Security](https://mlsanity.com/paper/2607.20255/) (2026)
- [From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation](https://mlsanity.com/paper/2609.22664/) (2026)
- [Toward Secure AI-Powered Penetration Testing Agents: Security Threats, Guardrails, and Architectural Perspectives](https://mlsanity.com/paper/2609.16694/) (2026)

## Abstract (arXiv)

Recent work on LLM-driven autonomous penetration testing reports promising results, but existing systems often bundle architectural, prompting, and tool-integration choices together. This makes it difficult to determine what is gained over a simple agent and harness. We present Cochise, a 630 LOC Python reference implementation for autonomous penetration-testing experiments. Cochise connects to a Linux execution host over SSH and supports attacking controlled target environments reachable from that jump host. The prototype implements a Planner--Executor architecture in which long-term state is maintained by the planner, while a ReAct-style executor issues commands over SSH and self-corrects based on command outputs. The scenario prompt can be adapted to different target environments. We evaluate the harness against a live third-party testbed, Game of Active Directory (GOAD). Cochise is intended not as a state-of-the-art penetration-testing agent, but as a reusable experimental infrastructure for comparing models, agent architectures, and penetration-testing traces. Alongside the prototype, we release replay and analysis tools: (i) cochise-replay for offline visualization of captured runs, (ii) cochise-analyze-logs and cochise-analyze-graphs for cost, token, duration, and compromise analysis, and (iii) a corpus of JSON trajectory logs from GOAD runs, so that researchers can study agent behavior without provisioning the 48--64 GB RAM / 190 GB storage testbed themselves. Tool demo video available at https://youtu.be/2mQimB1ufyI.