Agents of Chaos
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.
Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. First posted on arXiv 23 Feb 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 24.
24 papers.
Sort:
By area:
- 2026ClawSafety: "Safe" LLMs, Unsafe Agentscited by 15
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Studycited by 14
- 2026SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Docited by 4
- 2026Towards Security-Auditable LLM Agents: A Unified Graph Representationcited by 4
- 2026Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?cited by 4
- 2026Caging the Agents: A Zero Trust Security Architecture for Autonomous AI in Healthcarecited by 3
- 2026ClawTrap: A MITM-Based Red-Teaming Framework for Real-World OpenClaw Security Evaluationcited by 3
- 2026Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defensecited by 3
- 2026LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectioncited by 3
- 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Webcited by 2
- 2026SentinelAgent: Intent-Verified Delegation Chains for Securing Federal Multi-Agent AI Systemscited by 2
- 2026Improving the Efficiency of Language Agent Teams with Adaptive Task Graphscited by 2
12 more
- 2026Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Valuescited by 2
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agentscited by 2
- 2026Real-Time AI Service Economy: A Framework for Agentic Computing Across the Continuumcited by 1
- 2026Agent Meltdowns: The Road to Hell Is Paved with Helpful Agentscited by 1
- 2026Security of OpenClaw Agents: Fundamentals, Attacks, and Countermeasurescited by 1
- 2026Solipsistic Superintelligence is Unlikely to be Cooperativecited by 1
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substratescited by 1
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocolcited by 0
- 2026Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackerscited by 0
- 2026RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguardscited by 0
- 2026Agentic Societies Need a Social Harnesscited by 0
- 2026MiniRep: Robust Reputation-Based Aggregation for Multi-Agent Debatecited by 0
- 2026MiniRep: Robust Reputation-Based Aggregation for Multi-Agent Debate30 Sep 2026
- 2026Agentic Societies Need a Social Harness15 Sep 2026
- 2026RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards25 Aug 2026
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates1 Jul 2026
- 2026Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers19 Jun 2026
- 2026Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?4 Jun 2026
- 2026Solipsistic Superintelligence is Unlikely to be Cooperative2 Jun 2026
- 2026Security of OpenClaw Agents: Fundamentals, Attacks, and Countermeasures25 May 2026
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol23 May 2026
- 2026Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents18 May 2026
- 2026LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection18 May 2026
- 2026Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values11 May 2026
12 more
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents10 May 2026
- 2026Towards Security-Auditable LLM Agents: A Unified Graph Representation7 May 2026
- 2026Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs7 May 2026
- 2026Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defense4 May 2026
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study3 Apr 2026
- 2026SentinelAgent: Intent-Verified Delegation Chains for Securing Federal Multi-Agent AI Systems3 Apr 2026
- 2026ClawSafety: "Safe" LLMs, Unsafe Agents1 Apr 2026
- 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Web30 Mar 2026
- 2026SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Do28 Mar 2026
- 2026ClawTrap: A MITM-Based Red-Teaming Framework for Real-World OpenClaw Security Evaluation19 Mar 2026
- 2026Caging the Agents: A Zero Trust Security Architecture for Autonomous AI in Healthcare18 Mar 2026
- 2026Real-Time AI Service Economy: A Framework for Agentic Computing Across the Continuum5 Mar 2026
- 2026ClawSafety: "Safe" LLMs, Unsafe Agentscited by 15
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Studycited by 14
- 2026SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Docited by 4
- 2026Towards Security-Auditable LLM Agents: A Unified Graph Representationcited by 4
- 2026Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?cited by 4
- 2026Caging the Agents: A Zero Trust Security Architecture for Autonomous AI in Healthcarecited by 3
- 2026ClawTrap: A MITM-Based Red-Teaming Framework for Real-World OpenClaw Security Evaluationcited by 3
- 2026Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defensecited by 3
- 2026LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injectioncited by 3
- 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Webcited by 2
- 2026SentinelAgent: Intent-Verified Delegation Chains for Securing Federal Multi-Agent AI Systemscited by 2
- 2026Improving the Efficiency of Language Agent Teams with Adaptive Task Graphscited by 2
12 most cited of 19 All 19 (JSON)
- 2026MiniRep: Robust Reputation-Based Aggregation for Multi-Agent Debate30 Sep 2026
- 2026Agentic Societies Need a Social Harness15 Sep 2026
- 2026RePolicy: Reinforcement Learning for Safety-Policy Invocation in Agent Safeguards25 Aug 2026
- 2026Conversable Complexity: Agentic LLM Collectives as Interpretable Substrates1 Jul 2026
- 2026Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?4 Jun 2026
- 2026Security of OpenClaw Agents: Fundamentals, Attacks, and Countermeasures25 May 2026
- 2026LivePI: More Realistic Benchmarking of Agents Against Indirect Prompt Injection18 May 2026
- 2026Agent-ValueBench: A Comprehensive Benchmark for Evaluating Agent Values11 May 2026
- 2026Towards Security-Auditable LLM Agents: A Unified Graph Representation7 May 2026
- 2026Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs7 May 2026
- 2026Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defense4 May 2026
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study3 Apr 2026
12 newest of 19 All 19 (JSON)
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agentscited by 2
- 2026Agent Meltdowns: The Road to Hell Is Paved with Helpful Agentscited by 1
- 2026Solipsistic Superintelligence is Unlikely to be Cooperativecited by 1
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocolcited by 0
- 2026Solipsistic Superintelligence is Unlikely to be Cooperative2 Jun 2026
- 2026Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol23 May 2026
- 2026Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents18 May 2026
- 2026The Authorization-Execution Gap Is a Major Safety and Security Problem in Open-World Agents10 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
- 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
- 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
- 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents
- 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response
- 2026REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems
- 2026Autonomous Adversary: Red-Teaming in the age of LLM
- 2026The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey