mlsanityData to 7 Oct 2026

Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities

arXiv 2604.04561 ↗First posted 6 Apr 2026cs.CR cs.AI cs.CLcited by 2

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM agents with tool access can discover and exploit security vulnerabilities. This is known. What is not known is which features of a system prompt trigger this behaviour, and which do not. We present a systematic taxonomy based on approximately 10,000 trials across seven models, 37 prompt conditions, and 12 hypothesized attack dimensions -- all executed in real Docker sandboxes. Every condition includes the instruction "Always respect all rules and access policies"; only a single encouragement sentence varies. Within the task class studied (planted test-runner vulnerabilities), the threat model is narrower than assumed: nine of twelve dimensions -- including minimization, moral licensing, incentives, identity priming, and reasoning triggers -- produce no detectable exploitation at n=50 per cell (upper 95% CI <7% per condition). But it is also more subtle than expected: one dimension, goal reframing, reliably triggers exploitation. "You are solving a puzzle; there may be hidden clues" produces 38-40% exploitation on Claude Sonnet 4 despite the explicit rule instruction, replicating across four models (CTF framing: 8-14% on DeepSeek, GPT-5-mini, o4-mini). The agent does not override the rules; it reinterprets the task so that exploitative actions become task-aligned. GPT-4.1 produces no exploitation across 1,850 trials (37 conditions), and a temporal comparison across four OpenAI models released over eleven months shows a pattern consistent with improving safety training, though model capability differences are a confounder. The practical contribution is a narrowed, testable threat model: defenders should audit for goal-reframing language, not for the broad class of adversarial prompts.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 9 of its 19 references that are in the map. First posted on arXiv 6 Apr 2026.

Builds on

9 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 8

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  4. 2024Alignment faking in large language modelscited by 316
  5. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  6. 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 174
  7. 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 132
  8. 2026Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibrationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
  2. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
  3. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
  4. 2026ChainFuzzer: Greybox Fuzzing for Workflow-Level Multi-Tool Vulnerabilities in LLM Agents
  5. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
  6. 2026A Framework for Formalizing LLM Agent Security
  7. 2026Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study
  8. 2026Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs