mlsanityData to 7 Oct 2026

Agent threats, guardrails and governance

2,092 papers28 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: agents, security, attack, agent, safety

Key facts. Agent threats, guardrails and governance: 2,092 papers on arXiv since 2022 (data through 7 Oct 2026). 334 of its 2,092 mapped papers (16%) cite “AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents” (2024, arXiv 2406.13352). 289 of its 2,092 mapped papers (14%) cite “InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents” (2024, arXiv 2403.02691). 161 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

161 new papers, 17 Sep – 7 Oct; 28 this week.

  1. This week · 28
  2. From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage
  3. From Expected Harmfulness to Likelihood: A Probabilistic Reformulation of Jailbreaking LLM Agents
  4. Secure-CUA: Controlling Untrusted Influence in Computer-Use Agents
  5. RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment also fits: Multimodal and embodied agents
  6. Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files
  7. SpecGuard: Proving a Task Is Broken Before the Agent Cheats also fits: Software engineering coding agents
  8. SwarmReconGuard: Black-Box Detection of Distributed Collective Reconnaissance by Individually Benign-Looking Agent Populations
  9. How Fragile Is On-Device Language Model Safety? Localizing Safety-Critical Parameters for Sparse Fault Analysis
  10. Multi-Aspect Runtime Verification for Simulation-Based V&V of LLM-Enabled Autonomous Agents also fits: General LLM agent frameworks and benchmarks
  11. CredLeakBench: Evaluating Credential Leakage and Recovery in LLM Agents
All 161
  1. AdvSim2Real : Training Web Agents Against Adaptive Prompt Injection in a Web World Model also fits: Agent skill learning and evolution
  2. Semantic Behavioral Watermarking: Paraphrase-Robust and Forgery-Resistant Provenance for LLM Agents
  3. Beyond Corrected Memory: Execution Consistency in Multi-Agent Systems also fits: Software engineering coding agents
  4. POLAR: Ontology-Guided Risk Prevention for Tool-Calling LLM Agents
  5. DecepEval: A Benchmark for Evaluating Deception in LLM Agents also fits: Social simulation and multi-agent debate
  6. Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security
  7. Evaluate the Stack, Not the Layer: Do Deterministic and LLM Gates for Agent Actions Fail Independently? also fits: General LLM agent frameworks and benchmarks
  8. Lineage-Aware Memory Governance: A Derivation-Gated Framework for Privacy-Preserving Column-Level Access Control in Enterprise AI Agents
  9. Towards a Unified Misuse Monitoring Benchmark
  10. Before Agent Tells The Lie: Has Deception Already Been Represented? also fits: Social simulation and multi-agent debate
  11. ANT: A Multi-Granularity Network Traffic Dataset and Benchmark for Agents Behavior Auditing
  12. AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
  13. RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents
  14. Benchmarking Jailbreak Guardrails for Embodied Agents
  15. AgentSpy: Making AI Agent Behavior Observable
  16. Measurement-First Auditing of Agentic Leaderboards: Contamination Susceptibility, Matched-Control Re-evaluation, and Scorer Validation
  17. Topology-Conditioned Backdoors: Language Models That Insert Vulnerabilities When They Infer They Are in a Multi-Agent System
  18. Agentic-ZTA: A Multi-Agent Architecture for Autonomous Zero Trust Enforcement
  19. Earlier · 133
  20. AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search also fits: Web, search, and research agents
  21. G-CARB: Graph-Localized Conformal Agent Risk Budget for Compositional Harm
  22. Readable Before Actionable: Causal Tracing of Indirect Prompt Injection also fits: Agent skill learning and evolution
  23. APEX: Active Protection at Execution Boundaries for LLM Agents
  24. Lie Rarely, Lie Big: Stealthy Insider Attacks on LLM Robot Teams
  25. StegoMemory: Agentic Memory Acts as Covert Steganographic Channel
  26. From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agents
  27. TRIZa: Guiding Agent Evolution from Pattern Recognition to Solution Invention
  28. Quantifying Collusion Among Autonomous LLM Agents: A Statistical Analysis of the Collusion Wiki Incident
  29. The Same Zero: Why Identical ASR Can Imply Different Guarantees in LLM-Agent Security
  30. COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layers
  31. Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience
  32. Self-Propagating Misalignment in LLM Agents, and Why Auditing or Disabling Memory Is Not Enough
  33. From Requirements to Attack Trees: Grounded LLM Agents for Design-Time Security Review
  34. Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks
  35. Passing the Test You Trained On: Re-evaluating Prompt-Injection Detectors for LLM Agents
  36. EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents
  37. Securing Computer-Use Agents Against Branch Steering Attacks
  38. Beyond Predefined Sinks: Security-Aware Dependency Analysis for LLM Agents
  39. Positive-Unlabeled Learning for Agent Safety False Alarm Auditing
  40. A GHOST in Long-Horizon Agents: Governance Hazard from Overlooked Safety Constraints across Turns
  41. Prompted to Discriminate: Generalizing Malicious-Input Probes in the Wild
  42. MIRROR: Multipath Quorum Integrity for LLM Multi-Agent Communication
  43. KaliBench: A Fine-Grained Benchmark for Cybersecurity Tool Use on Kali Linux with Runtime-Free Verifiable Rewards
  44. OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents
  45. PACE: Provenance-Aware Capability Enforcement for Tool-Using LLM Agents
  46. AuraForge: Scaling Security Supervision for Training Coding Agents
  47. Memetic Trojans: Social Contagions as Carriers of Adversarial Payloads in Agent Networks
  48. Representation Transitions Reveal Emerging Safety Risks in Multi-Turn LLM Agents
  49. Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents
  50. Speculative Safety Honeypot: Toward Proactive Defense Against Multi-turn Agent Attacks
  51. ActionGuard: Tool Call Authorization under Poisoned Skills
  52. Hiding in Plain Sight: Decoupling Pretext from Actuation for Skill Poisoning in LLM Agents
  53. MiniRep: Robust Reputation-Based Aggregation for Multi-Agent Debate also fits: Social simulation and multi-agent debate
  54. MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models also fits: General LLM agent frameworks and benchmarks
  55. Can Agents Trust Their Skills? Uncovering Unsafe Chains of Trust in Skill-Based LLM Agents
  56. Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems
  57. Cheap to Hypothesize, Costly to Verify: The Defense Surface of Agentic Vulnerability Discovery
  58. HARDE: Optimizing Agent Harnesses for Runtime Risk Detection and Execution Control
  59. Concealing LLM-Based Multi-Agent Topology via Phantom Structure Injection
  60. ToolFence: Fine-Grained Authorization for Secure Tool-Using LLM Agents
  61. SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs
  62. SafeCoEvo: Co-Evolving Safety Harnesses and Guards for LLM Agents at Test-Time
  63. Divide and Inject: Can Agents Reconstruct an Indirect Prompt Injection from Fragments?
  64. CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
  65. LLMs Learn to Evade Latent Monitors from Prior Feedback Alone
  66. CoSec: Benchmarking Agent Security in Communities
  67. Auditing Agent Actions through Query-Conditioned Attribution also fits: Web, search, and research agents
  68. Evaluating Bounded Autonomy in Regulated Agentic AI: A Diagnostic Harness with Constitutional Rewards, Escalation Labels, and Runtime Governance also fits: Software engineering coding agents
  69. SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety
  70. PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agents
  71. Same Bytes, Different Authority: Reserved-Token Representations in Chat-Template Prompt Injection
  72. MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?
  73. Share-Borne AI Virus: Memory-Hopping Attacks Across LLM Agents also fits: Agent memory systems
  74. Before Acting, Change the State: Prospective State Intervention for Web Agents under Deceptive Interfaces
  75. Jailbreak Context Lingers: Divergent Safety Routing and Its Cross-Task Predictability in Tool Agents
  76. SEAD: A State-Based Perspective on Attack and Defense in Tool-Using Agents
  77. CoDeL: Co-Evolutionary Defense against Indirect Prompt Injection in LLM-based Agents
  78. PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety
  79. Certified Multi-Source Integrity for Structured Agent Actions
  80. AdaGuard: An Adaptive Guard Model with User-defined Policies also fits: Reinforcement learning for LLM agents
  81. Coding Agents Aren't Enough! Evaluating an Enterprise Security Brain for Agentic Cloud Investigations
  82. SINGED: Correct Outputs Do Not Certify Safe Execution in LLM Agents
  83. Agentic Commerce Bench: Measuring Fraud Detection for Agents That Spend Money also fits: Financial trading and forecasting agents
  84. When Consent Outlives Context: Residual Authority Replay in Long-Lived Agents
  85. SecProbe: Adaptive Evaluation of Coding Agents on Cybersecurity Vulnerabilities
  86. AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents also fits: Tool learning and function calling
  87. Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation
  88. API Secrets Should Never Become Tokens in the LLM's Vocabulary: A Threat Analysis of API Credential Handling in LLM Agent Systems and an Empirical Evaluation of a Vault-Mediated Execution Boundary
  89. Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal States
  90. Boundary-State Control for Tool-Using Language-Model Agents: Commit-Time Consistency under State Drift
  91. Backdoor in the Loop: Compromising Agentic Search via Malicious Retrievers
  92. Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time
  93. CAIRN: Dynamic Fact-Intent DAGs for Multi-Agent Exploration also fits: General LLM agent frameworks and benchmarks
  94. Silent Failures in Agentic Security Evaluation: A Validated Harness for Tool-Call Mediation Under Indirect Prompt Injection
  95. Business Compromise Detection with Agentic AI and LLM-driven Knowledge Discovery also fits: Financial trading and forecasting agents
  96. Trust the Brand, Lose Control: How Identity Hijacks LLM Agent Orchestration
  97. "You're Right, Let Me Fix It": How LLM Agents Damage Correct Work When Falsely Accused also fits: Agent skill learning and evolution
  98. Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructions
  99. CyberClear: A Benchmark for LLM Agent Systems on APT Attack Chain Provenance
  100. AI Harness: Certification under Proposal-Conditioned Information for Foundation-Model Agents
  101. SilentCall: Hidden Tool-Call Backdoors in Open-Weight Agents, and How to Catch Them
  102. A Safety-Bounded SDC-to-MCP Gateway for Medical AI Agents also fits: Medical and scientific domain agents
  103. AgentXploit: Autonomous Repository-to-Runtime Red-Teaming for AI Agents
  104. Bounded Autonomy and Verifiable Safety for Agentic AI Enabled Automation also fits: General LLM agent frameworks and benchmarks
  105. Subjects, Not Authors: The Authorship Hazard in Agentic Dataspaces also fits: Web, search, and research agents
  106. Stealth Apart, Harm Together: Skill Cascading Attacks on Skill-Based Agent Systems also fits: Agent skill learning and evolution
  107. LLM Agents Can Easily Tamper With Their Own Traces
  108. Hard Stop: Kernel-Level Preemption and Containment for Rogue Agentic Execution
  109. Demystifying Agent Skills for Smart Contract Auditing: Design, Effectiveness, Behavioral Impact also fits: Software engineering coding agents
  110. Calibrated Decision Models for Autonomous Penetration-Testing Harnesses: JEV and Laya as System One Decision Layers for LLM-Driven Pentest Agents
  111. On the Effectiveness of Kernel-Level Evidence for Agent Security
  112. Codetta: High-Capacity, Keyless, and Undetectable Multi-Agent Collusion
  113. Blockchain-Enabled Artificial Intelligence and AI Agents for Secure Data Sharing and Cybersecurity Applications
  114. Reward Hacking Challenges Oversight of Autonomous Research Agents
  115. Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions
  116. Persistent Billable State: Denial-of-Wallet Attacks and Defenses in Tool-Calling LLM Agents
  117. Where Cyber Agents Struggle: Bottleneck Analysis of Multi-Stage LLM Agents
  118. Who Is Behind the Harness? Fingerprinting LLMs through Agentic Behavior also fits: Software engineering coding agents
  119. PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety also fits: General LLM agent frameworks and benchmarks
  120. Control-Token Injection Suppresses Chain-of-Thought and Defeats Reasoning-Based Oversight in Tool-Using Agents
  121. Ajar: Measuring Open Privilege in Agent Defenses
  122. A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem
  123. How Strongly Should Task State Influence an LLM Agent? also fits: General LLM agent frameworks and benchmarks
  124. Indirect tipping: a social attack surface in AI agent populations
  125. Attack Success Rate Is Not a Number: On Measurement Validity in Agentic AI Security Evaluation
  126. DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security
  127. ActGov: Governing LLM Agent Actions via Policy-Constrained Validation
  128. Dissecting Agentic Forensics: The Role of Triage, Prompting, and Evidence Arbitration in Open-World Fake Image Detection also fits: Multimodal and embodied agents
  129. Anticipatory Human Oversight of Agentic AI: A Philosophical Account also fits: General LLM agent frameworks and benchmarks
  130. LeaseGuard: Incumbent-Preserving Admission Control for Privileged LLM Agents
  131. MobileCybench: Evaluating Agent Vulnerability Discovery via Executable Probes
  132. Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control also fits: Financial trading and forecasting agents
  133. Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challenges
  134. When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systems
  135. Runtime Authorization Consistency Checking for MCP-based Agentic Workflows also fits: Software engineering coding agents
  136. Environment Steering: Using Data Flow Control to Improve Agent Utility and Safety
  137. Triggers and Diagnostics for LLM-Based Interpretability Failures in Active Inference Agents also fits: General LLM agent frameworks and benchmarks
  138. When Agentic Trust Crosses Organizational Boundaries: Structural Externalization and a Reference Model for Trust Evidence
  139. Beyond Single-Model Injection: A Threat Model and Defense Architecture for Prompt Injection in Multi-Agent Systems
  140. NostrAgent: A Decentralized Identity and Delegation Architecture for Sovereign Agentic Systems
  141. The Price of Safety: Benign-Case Utility and Token Overhead of Memory-Poisoning Defenses in LLM Agents
  142. SelfOp: An Optimization Algorithm for Self-Improving Security Agents
  143. MATE: Policy-Aware Security Auditing for Mobile Agents via Synthesis-Driven Trajectory Learning
  144. Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems
  145. From Capability to Assurance in Autonomous Penetration-Testing Harnesses: A Framework and Reference Implementation
  146. Zero-Trust Authorization and Discovery for Enterprise MCP
  147. Defusing Explosive Prompts: Understanding and Preventing Trigger-Based Prompt Injections in LLM Agents
  148. CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents
  149. LEGIT: Credentialing Protocol for Trustworthy AI Agent Marketplaces
  150. AI-GRACE: A Use-Case Operationalization Framework for Agentic AI: From Organizational Objectives and Obligations to Deployment Capabilities and Architecture also fits: Financial trading and forecasting agents
  151. Coding Agents with an Obstacle-Aware Harness for Safe Robot Manipulation also fits: General LLM agent frameworks and benchmarks
  152. Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents

Foundations

Most cited by papers here

  1. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents2024cited by 334 here
  2. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents2024cited by 289 here
  3. AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases2024cited by 243 here
  4. Identifying the Risks of LM Agents with an LM-Emulated Sandbox2023cited by 242 here
  5. Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents2024cited by 227 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 299 here
  2. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 159 here
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 182 here
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 209 here
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 142 here
  6. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 123 here
  7. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 141 here
17 more it builds on, not in this chain
  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 148 here
  2. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 108 here
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 172 here
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 186 here
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 125 here
  6. 2023AgentBench: Evaluating LLMs as Agentscited by 112 here
  7. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 242 here
  8. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 289 here
  9. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 138 here
  10. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 117 here
  11. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agentscited by 334 here
  12. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 243 here
  13. 2024EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakagecited by 105 here
  14. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 227 here
  15. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 212 here
  16. 2025Defeating Prompt Injections by Designcited by 141 here
  17. 2025WASP: Benchmarking Web Agent Security Against Prompt Injection Attackscited by 101 here

The 24 most cited of 3,006.

Browse

30 of 2,092, ranked 6–35

  1. 6AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents2024212
  2. 7Defeating Prompt Injections by Design2025141
  3. 8Agent-SafetyBench: Evaluating the Safety of LLM Agents2024141
  4. 9R-Judge: Benchmarking Safety Risk Awareness for LLM Agents2024123
  5. 10GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning2024117
  6. 11EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage2024105
  7. 12WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks2025101
  8. 13AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents202595
  9. 14Prompt Injection Attack to Tool Selection in LLM Agents202588
  10. 15Memory Injection Attacks on LLM Agents via Query-Only Interaction202581
All 30
  1. 16Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents202478
  2. 17ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning202577
  3. 18IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems202477
  4. 19Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems202475
  5. 20Red-Teaming LLM Multi-Agent Systems via Communication Attacks202571
  6. 21BadAgent: Inserting and Activating Backdoor Attacks in LLM Agents202471
  7. 22Dissecting Adversarial Robustness of Multimodal LM Agents202468
  8. 23Attacking Vision-Language Computer Agents via Pop-ups202468
  9. 24Progent: Securing AI Agents with Privilege Control202566
  10. 25OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents202566
  11. 26Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents202565
  12. 27The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents202464
  13. 28LlamaFirewall: An open source guardrail system for building secure AI agents202562
  14. 29Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast202461
  15. 30The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies202461
  16. 31RedCode: Risky Code Execution and Generation Benchmark for Code Agents202458
  17. 32A Survey on Trustworthy LLM Agents: Threats and Countermeasures202557
  18. 33Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification202457
  19. 34Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training202456
  20. 35SafeArena: Evaluating the Safety of Autonomous Web Agents202553