# Agent threats, guardrails and governance Area: [LLM agents and tool use](https://mlsanity.com/agents/index.md). HTML: https://mlsanity.com/agents/agent-threats-guardrails-and-governance/ ## Key facts - Agent threats, guardrails and governance: 2,092 papers on arXiv since 2022 (data through 14 Oct 2026). - 334 of its 2,092 mapped papers (16%) cite “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents” (2024, arXiv 2406.13352). - 289 of its 2,092 mapped papers (14%) cite “InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents” (2024, arXiv 2403.02691). - 205 new papers 17 Sep–14 Oct. ## Most cited here - [AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents](https://mlsanity.com/paper/2406.13352/) (2024), cited by 334 papers here - [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024), cited by 289 papers here - [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024), cited by 243 papers here - [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023), cited by 242 papers here - [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024), cited by 227 papers here - [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024), cited by 212 papers here - [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024), cited by 141 papers here - [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025), cited by 141 papers here - [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024), cited by 123 papers here - [GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning](https://mlsanity.com/paper/2406.09187/) (2024), cited by 117 papers here ## New (17 Sep–14 Oct) - [StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning](https://mlsanity.com/paper/2610.20960/) (2026-10-14) - [INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems](https://mlsanity.com/paper/2610.20776/) (2026-10-14) - [From Training to Escalation: Forensic Analysis and Local Recreation of RL/RLHF-Induced Privilege Escalation in AI Agents (CVE-2026-65616)](https://mlsanity.com/paper/2610.20437/) (2026-10-14) - [DarKnight: Retrieval-Guided Agentic Framework for Breaking Dark Web CAPTCHA Gates](https://mlsanity.com/paper/2610.19830/) (2026-10-14) - [One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AI](https://mlsanity.com/paper/2610.19818/) (2026-10-14) - [MCAIR: Monitoring and Correcting Agent Unfaithfulness via Internal Representations](https://mlsanity.com/paper/2610.19679/) (2026-10-14) - [PFedMA: Who Should Join Next as Federated Multi-Agent Orchestration Evolves?](https://mlsanity.com/paper/2610.19458/) (2026-10-14) - [AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety](https://mlsanity.com/paper/2610.19422/) (2026-10-14) - [AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents](https://mlsanity.com/paper/2610.19158/) (2026-10-13) - [CyberArena: Benchmarking AI Agents in Live Symmetric Attack-and-Defense Combat](https://mlsanity.com/paper/2610.19005/) (2026-10-13) - [Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries](https://mlsanity.com/paper/2610.18948/) (2026-10-13) - [Towards Safe Autonomy for Computer Use](https://mlsanity.com/paper/2610.18508/) (2026-10-13) - [Kairos: Transactional Security Enforcement at the System Boundary for LLM Agents](https://mlsanity.com/paper/2610.18457/) (2026-10-13) - [Jev-Guard: Fast Routing and Auditing Agent Actions over Editable Policy Trees](https://mlsanity.com/paper/2610.18326/) (2026-10-13) - [Encrypted Skills Can Become New Security Exploits If AI Agents Know How to Decipher](https://mlsanity.com/paper/2610.18262/) (2026-10-13) - [Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests](https://mlsanity.com/paper/2610.18007/) (2026-10-13) - [ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?](https://mlsanity.com/paper/2610.17837/) (2026-10-13) - [Empowering Edge Agents: A Systematic Analysis of Edge-Cloud Collaboration in LLM Agent Execution](https://mlsanity.com/paper/2610.17816/) (2026-10-13) - [Evidence-First Agentic Verification](https://mlsanity.com/paper/2610.17724/) (2026-10-13) - [Welcome to the Age of Agent Ads: Using On-Policy Attacks to Steer Agentic Decisions](https://mlsanity.com/paper/2610.17195/) (2026-10-12) - [CWE-bench: Refocusing Agentic Security from Exploitation to Defense](https://mlsanity.com/paper/2610.17020/) (2026-10-12) - [Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization](https://mlsanity.com/paper/2610.16838/) (2026-10-12) - [AuthGateBench: Benchmarking LLMs as Authorization Gate](https://mlsanity.com/paper/2610.16426/) (2026-10-12) - [TRAP: Tool-result Replay Attack for Probing Misbinding in LLM Agents](https://mlsanity.com/paper/2610.16333/) (2026-10-12) - [Phantom Merge: When Your Large Language Model Agents Pick One but Tell You About Another](https://mlsanity.com/paper/2610.16141/) (2026-10-12) - [When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems](https://mlsanity.com/paper/2610.16039/) (2026-10-12) - [Revocation Is Not Complete at Receipt: An Executable Outcome Contract for Distributed Agent-Tool Gateways](https://mlsanity.com/paper/2610.15991/) (2026-10-11) - [Scalable Weak-to-Strong Verifier Hardening for Agent Environments](https://mlsanity.com/paper/2610.15924/) (2026-10-11) - [Agent-Canary-Mas: End-to-End Evaluation of Protocol Risk in Long-Horizon Multi-Agent Systems](https://mlsanity.com/paper/2610.15529/) (2026-10-11) - [From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents](https://mlsanity.com/paper/2610.15390/) (2026-10-11)