PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Tool-using large language model (LLM) agents turn generated text into real side effects, so poisoned tool metadata, retrieved pages, memory, and reusable skills can steer the next call. Vetting an artifact before admission does not settle this. A safe variant and a leaking variant can produce the same admission evidence, and a sound gate then cannot relax that site for either. We make that condition precise, which leaves the last boundary a deployment can still act on. We present Provenance-Aware Capability Enforcement (PACE), which mediates every tool call immediately before it executes. Path confinement proposes an executable cut of represented influence paths, while capability and effect verification checks schema-defined effects against authority compiled from the authenticated request. We distinguish the certified execution contract from the evaluated configuration, which can restore an authorized call after a proposed block or apply a declared repair. Confinement requires the final action to preserve the certified cut. On eight executable agent-security benchmarks with three target-model families, the evaluated configuration gives strictly lowest attack success in 62 of 79 eligible attack columns and ties in 14; full-benchmark native utility loses at most three points relative to the undefended agent. A complete ablation over 1167 paired cases attributes most security gains to effect verification and refusal control to boundary adaptation. A reduced-scale adaptive search succeeds on 0/30 out-of-authority targets against the defense.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 49 of its 77 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
49 of 77 references.
Closest ancestors · 27
Same direction, newest first
- 2026Context-to-Execution Integrity for LLM Agents7 Jul
- 2026Adaptive Evaluation of Out-of-Band Defenses Against Prompt Injection in LLM Agents25 Jun
- 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agents26 May
- 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneck11 May
- 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection13 Apr
- 2026SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement5 Apr
- 2026Formal Policy Enforcement for Real-World Agentic Systems18 Feb
- 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents13 Feb
- 2026From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent9 Feb
- 2025MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents14 Oct
- 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents21 Aug
- 2025MCPSecBench: A Systematic Security Benchmark and Playground for Testing Model Context Protocols17 Aug
15 more
- 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
- 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents13 Jun
- 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
- 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks22 Apr
- 2025Progent: Securing AI Agents with Privilege Control16 Apr
- 2025Defeating Prompt Injections by Design24 Mar
- 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
- 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage13 Feb
- 2025MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents7 Feb
- 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents21 Dec
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 22
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
10 more
- 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
- 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
- 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
- 2025Securing AI Agents with Information-Flow Controlcited by 108
- 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
- 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
- 2025Contextual Agent Security: A Policy for Every Purposecited by 55
- 2026AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?cited by 45
- 2025MCIP: Protecting MCP Safety via Model Contextual Integrity Protocolcited by 33
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Context-Fractured Decomposition Attacks on Tool-Using LLM Agents: Exploiting Artifact Provenance Gaps
- 2026Capability Gates Are Not Authorization: Confused-Deputy Failures in LLM Agent Frameworks
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
- 2026ToolFence: Fine-Grained Authorization for Secure Tool-Using LLM Agents
- 2026WebMCP-Phalanx: Enforcing and Characterizing Trust Boundaries for Browser-Integrated LLM Agents
- 2026Safeguarding LLM Agents from Misalignment through Provenance Analysis
- 2026VIGIL: Defending LLM Agents Against Tool Stream Injection via Verify-Before-Commit
- 2026Beyond Max Tokens: Stealthy Resource Amplification via Tool Calling Chains in LLM Agents