mlsanityData to 7 Oct 2026

Agent-Sentry: Bounding LLM Agents via Execution Provenance

arXiv 2603.22868 ↗First posted 24 Mar 2026cs.CR cs.AIcited by 13

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Agentic computing systems, while immensely capable, raise serious security, privacy, and safety concerns. A key issue is that the full set of functionalities offered by these systems, combined with their probabilistic execution flows, is not known beforehand. Given this lack of characterization, it is challenging to validate whether a system has successfully carried out the user's intended task or instead executed irrelevant actions, potentially as a consequence of compromise. We present Agent Sentry, a runtime defense that learns a bound on an agent's benign execution from prior legitimate executions and flags any action that falls outside this bound. Agent Sentry layers three complementary checks: a structural classifier over the sequence of actions and the provenance of each function's arguments; a deterministic allowlist check over sensitive argument values; and an LLM judge, invoked only on the residual of actions where the first two checks cannot safely decide between a legitimate new request and a carefully crafted injection. We demonstrate the effectiveness of Agent Sentry in AgentDojo and AgentDyn by blocking 94.3% of successful injections while allowing 95.1% of benign executions, without modifying the agent, its tools, or the LLM.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 34 of its 74 references that are in the map. First posted on arXiv 24 Mar 2026.

Builds on

34 of 74 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injection2 Aug
  2. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents13 Jun
  3. 2025Progent: Securing AI Agents with Privilege Control16 Apr
  4. 2025Defeating Prompt Injections by Design24 Mar
  5. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  6. 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
  7. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  8. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 26

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  8. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  9. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  10. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  12. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
14 more
  1. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  2. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 117
  3. 2025Securing AI Agents with Information-Flow Controlcited by 108
  4. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 91
  5. 2025A Survey of LLM-based Agents in Medicine: How far are we from Baymax?cited by 84
  6. 2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 82
  7. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
  8. 2024Building Guardrails for Large Language Modelscited by 61
  9. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 61
  10. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  11. 2025LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judgecited by 33
  12. 2023Kairos: Practical Intrusion Detection and Investigation using Whole-system Provenancecited by 30
  13. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  14. 2024Purple-teaming LLMs with Adversarial Defender Trainingcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agentscited by 29
  2. 2026SoK: Security of Autonomous LLM Agents in Agentic Commercecited by 8
  3. 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 8
  4. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 8
  5. 2026Towards Security-Auditable LLM Agents: A Unified Graph Representationcited by 4
  6. 2026Self-Evolving Agents as Dynamic Graph Transformation: A Survey and New Perspectivecited by 4
  7. 2026PocketAgents: A Manifest-Driven Library of Autonomous Defense Agentscited by 0
  8. 2026The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilitiescited by 0
  9. 2026Token-Flow Firewall: Semantic Runtime Auditing for Persistent AI Agentscited by 0
  10. 2026Grading the Narrators: An Isnad-Rijal Framework for Claim-Level Provenance in Multi-Agent Knowledge Systemscited by 0
  11. 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agentscited by 0
  12. 2026Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agentscited by 0
1 more
  1. 2026On the Effectiveness of Kernel-Level Evidence for Agent Securitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injection
  2. 2026Securing LLM Agents Need Intent-to-Execution Integrity
  3. 2026AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
  4. 2026Position: A Three-Layer Probabilistic Assume-Guarantee Architecture Is Structurally Required for Safe LLM Agent Deployment
  5. 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systems
  6. 2026SafeAgent: A Runtime Protection Architecture for Agentic Systems
  7. 2026When Tool Outputs Become Commands: Separating Action Induction from Runtime Authorization in Tool-Augmented LLM Agents
  8. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents