# Agent Hacks Agent: Autoresearch for Production-Agent Red-Teaming

arXiv: https://arxiv.org/abs/2607.11698 · HTML: https://mlsanity.com/paper/2607.11698/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 52 of its 62 references that are in the map.
- First posted on arXiv 13 Jul 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 52)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024)
- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily](https://mlsanity.com/paper/2311.08268/) (2023)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs](https://mlsanity.com/paper/2410.05295/) (2024)
- [Where LLM Agents Fail and How They can Learn From Failures](https://mlsanity.com/paper/2509.25370/) (2025)
- [The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections](https://mlsanity.com/paper/2510.09023/) (2025)
- [Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents](https://mlsanity.com/paper/2503.00061/) (2025)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [RedCode: Risky Code Execution and Generation Benchmark for Code Agents](https://mlsanity.com/paper/2411.07781/) (2024)
- [AgentRx: Diagnosing AI Agent Failures from Execution Trajectories](https://mlsanity.com/paper/2602.02475/) (2026)
- [AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?](https://mlsanity.com/paper/2602.03117/) (2026)
- [Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework](https://mlsanity.com/paper/2601.11893/) (2026)
- [AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security](https://mlsanity.com/paper/2601.18491/) (2026)
- [RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent](https://mlsanity.com/paper/2407.16667/) (2024)
- [AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration](https://mlsanity.com/paper/2503.15754/) (2025)
- [Measuring Agents in Production](https://mlsanity.com/paper/2512.04123/) (2025)
- [ATBench: A Diverse and Realistic Agent Trajectory Benchmark for Safety Evaluation and Diagnosis](https://mlsanity.com/paper/2604.02022/) (2026)
- [AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification](https://mlsanity.com/paper/2602.22724/) (2026)
- [ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?](https://mlsanity.com/paper/2605.11086/) (2026)
- [MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attacks](https://mlsanity.com/paper/2602.09222/) (2026)
- [AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agents](https://mlsanity.com/paper/2604.02947/) (2026)
- [CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures](https://mlsanity.com/paper/2605.25338/) (2026)

## Built on by (most cited first, 2 of 2)

- [HarnessSafe: Evaluating Safety Across Persistent Carriers in Agent Harnesses](https://mlsanity.com/paper/2608.06984/) (2026)
- [Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents](https://mlsanity.com/paper/2608.12851/) (2026)

## Abstract (arXiv)

Production LLM agents such as Claude Code and Codex can modify files and execute commands, so safety failures become real destructive actions. Automatic red-teaming lets safety teams test beyond static suites at the pace of deployment updates. Existing methods retain successful attacks but not why each attack succeeded, so after a failed reuse testers cannot tell whether the weakness is gone or the attack no longer fits. We instead retain vulnerability concepts, each stating why an attack succeeds, the condition that enables it, and the evidence that would refute it. Agent Hacks Agents (AHA) discovers these concepts with a Karpathy-style autoresearch loop that tests falsifiable hypotheses on agent trajectories. Repeatedly confirmed concepts enter a vulnerability concept graph that links related weaknesses. Across 18 settings of three scenarios, three victim models, and two agents, the concepts divide into eight families. A shared core recurs across agents, while diverse families appear only in specific settings. On held-out instances the concepts reach a 47.0% attack success rate (ASR) against 32.8% for the strongest baseline, which uses more discovery queries. Beyond the discovery setting, the concepts support two kinds of reuse. Generalization carries concepts to new victims, scenarios, and harnesses, and coordination joins graph-linked concepts into stronger attacks on two of three scenarios. For defense, patching each concept's enabling condition lowers AgentHazard ASR by 41.11 points. The concepts explain why production agents fail and where to repair the agents. Our code is in https://github.com/henrymao2004/Auto-research-red-teaming.