Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents arbitrate among instructions from system prompts, users, memory, and tools, but this arbitration cannot be assumed to enforce trust boundaries. We identify a recognition-enforcement gap: source-format features (role-template position, channel metadata, formatting cues) are linearly decodable from model activations, and models can explicitly identify forged authority when prompted, yet some configurations still produce the conflicting tool call. We use "recognition" in this specific decodable-source-format-plus-verbalized-detection sense; crossed-probe controls show it is not a unified abstract trust representation. The gap is not an immutable property of model weights. Restrictive policies and diverse prompts can eliminate execution on the same models, while permissive configurations and particular prompt-model pairs yield deterministic failures. Across a fleet evaluation (authority spoofing: 46 model endpoints across 6 vendors including open-weight; memory conflict: 48 models), average execution under diverse novel attacks is 1.21% [0.5-2.1%] (model-clustered CI over 14,294 spoofed trials from 29 models), but vulnerability is concentrated in reproducible cells and shifts across deployment windows (up to 47pp within-window per-fingerprint range). Prompt-layer defenses likewise fail to generalize across models and adaptive formulations. We therefore treat model self-arbitration as a capability rather than a security boundary and implement an external reference monitor combining authenticated source routing with capability-gated tool execution. It deterministically rejects all tested forged, tampered, replayed, and unsigned requests while preserving legitimate operations. A separate adaptive red-team found one implementation flaw (a since-patched clock-skew admission), not a cryptographic bypass. Secure agents require external enforcement, not merely better recognition.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 14 of its 20 references that are in the map. First posted on arXiv 28 Aug 2026.
Builds on
14 of 20 references.
Closest ancestors · 6
Same direction, newest first
- 2026Forensic Trajectory Signatures for Agent Memory Poisoning Detection29 Jun
- 2026Injection-Execution Dissociation: A Mechanistic Evaluation of Persistent Memory Attacks and Defenses in Stateful LLM Agents8 May
- 2025Defeating Prompt Injections by Design24 Mar
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 8
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2024Knowledge Conflicts for LLMs: A Surveycited by 283
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 267
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2025Control Illusion: The Failure of Instruction Hierarchies in Large Language Modelscited by 33
- 2026Defenses Against Prompt Attacks Learn Surface Heuristicscited by 11
- 2026Online Shift Detection and Conformal Adaptation for Deployed Safety Classifierscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
- 2026Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agents
- 2026Mapping the Exploitation Surface: A 10,000-Trial Taxonomy of What Makes LLM Agents Exploit Vulnerabilities
- 2026Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
- 2026Skills as Verifiable Artifacts: A Trust Schema and a Biconditional Correctness Criterion for Human-in-the-Loop Agent Runtimes
- 2026Behavioral Integrity Verification for AI Agent Skills
- 2026Auditing Provenance Sensitivity in LLM Agent Action Selection