Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents with tool access can discover and exploit security vulnerabilities. This is known. What is not known is which features of a system prompt trigger this behaviour, and which do not. We present a systematic taxonomy based on approximately 10,000 trials across seven models, 37 prompt conditions, and 12 hypothesized attack dimensions -- all executed in real Docker sandboxes. Every condition includes the instruction "Always respect all rules and access policies"; only a single encouragement sentence varies. Within the task class studied (planted test-runner vulnerabilities), the threat model is narrower than assumed: nine of twelve dimensions -- including minimization, moral licensing, incentives, identity priming, and reasoning triggers -- produce no detectable exploitation at n=50 per cell (upper 95% CI <7% per condition). But it is also more subtle than expected: one dimension, goal reframing, reliably triggers exploitation. "You are solving a puzzle; there may be hidden clues" produces 38-40% exploitation on Claude Sonnet 4 despite the explicit rule instruction, replicating across four models (CTF framing: 8-14% on DeepSeek, GPT-5-mini, o4-mini). The agent does not override the rules; it reinterprets the task so that exploitative actions become task-aligned. GPT-4.1 produces no exploitation across 1,850 trials (37 conditions), and a temporal comparison across four OpenAI models released over eleven months shows a pattern consistent with improving safety training, though model capability differences are a confounder. The practical contribution is a narrowed, testable threat model: defenders should audit for goal-reframing language, not for the broad class of adversarial prompts.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 9 of its 19 references that are in the map. First posted on arXiv 6 Apr 2026.
Builds on
9 of 19 references.
Closest ancestors · 1
Same direction, newest first
Other references · 8
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
- 2024Alignment faking in large language modelscited by 316
- 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
- 2026Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibrationcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
- 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
- 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
- 2026A Framework for Formalizing LLM Agent Security
- 2026Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study
- 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs