Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As multi-agent systems enter high-stakes domains, the possibility that agents may circumvent safety boundaries is a growing concern. Prior work has examined this risk primarily in adversarial settings, where agents are instructed or rewarded to communicate covertly and evade oversight. We show that benign agents can cross the same boundaries without adversarial incentives. We emulate a software-engineering workflow in which a planner represents a company hiring an external developer. The planner writes requirements and holds a company credential it is instructed not to disclose to the developer; a monitor screens their exchanges. Seven of nine tested frontier models disguise the credential in their requirements to help the developer recover it while evading the monitor, even after completing their assigned objective. For example, across 6,000 episodes with DeepSeek-V4-Pro, the planner attempts concealment in 16.9%; in 0.9%, the credential evades the monitor and is recovered and used by the developer. Despite the low per-episode rate, the risk compounds across repeated exchanges: at 0.9%, 105 independent episodes carry a 61.3% chance of at least one breach. Our analysis suggests that the planner misinterprets disguised disclosure as helping the developer and reads the nondisclosure rule as banning plaintext, not character codes or riddles. We call this behavior covert assistance. Monitoring favors concealment, while concealment gives way to direct disclosure when the recipient is presented as a human rather than an AI agent, so leakage persists. Giving the monitor the credential improves detection but requires trusting it with the secret. These risks, in models already used for software engineering, challenge oversight to distinguish authorized cooperation from task-advancing assistance that crosses safety boundaries.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 63 of its 84 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
63 of 84 references.
Closest ancestors · 17
Same direction, newest first
- 2026Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations: A Black-Box, Cross-Principal Approach3 Aug
- 2026Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems25 Jun
- 2026An Application-Layer Multi-Modal Covert-Channel Reference Monitor for LLM Agent Egress20 May
- 2026PRISM: Generation-Time Detection and Mitigation of Secret Leakage in Multi-Agent LLM Pipelines11 May
- 2026The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans26 Apr
- 2026How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study3 Apr
- 2026Detecting Multi-Agent Collusion Through Multi-Agent Interpretability1 Apr
- 2026AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems12 Feb
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework17 Jan
- 2025Agentic Misalignment: How LLMs Could Be Insider Threats5 Oct
- 2025Audit the Whisper: Detecting Steganographic Collusion in Multi-Agent LLMs5 Oct
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
5 more
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganography12 Feb
- 2024LLM Agents can Autonomously Hack Websites6 Feb
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 46
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
34 more
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2023A Watermark for Large Language Modelscited by 725
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
- 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
- 2018AI safety via debatecited by 400
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2024Alignment faking in large language modelscited by 360
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 258
- 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safetycited by 249
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
- 2022Measuring Progress on Scalable Oversight for Large Language Modelscited by 225
- 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
- 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2020Open Problems in Cooperative AIcited by 168
- 2025Multi-Agent Risks from Advanced AIcited by 163
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
- 2023Tensor Trust: Interpretable Prompt Injection Attacks from an Online Gamecited by 145
- 2024AI Sandbagging: Language Models can Strategically Underperform on Evaluationscited by 145
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
- 2023Scheming AIs: Will AIs fake alignment during training in order to get power?cited by 97
- 2025Stress Testing Deliberative Alignment for Anti-Scheming Trainingcited by 86
- 2019Neural Linguistic Steganographycited by 78
- 2023Preventing Language Models From Hiding Their Reasoningcited by 65
- 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 55
- 2022Perfectly Secure Steganography Using Minimum Entropy Couplingcited by 44
- 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 44
- 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 18
- 2020Near-imperceptible Neural Linguistic Steganography via Self-Adjusting Arithmetic Codingcited by 17
- 2026Evaluating and Understanding Scheming Propensity in LLM Agentscited by 14
- 2026A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoringcited by 8
- 2026AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Ratescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026When AI Finds Hidden Messages, Does It Report?cited by 0
- 2026When AI Finds Hidden Messages, Does It Report?7 Oct 2026
Similar, not cited
Close in wording, no citation either way
- 2026Instrumental Monitor Evasion Emerges Under Ordinary Task Pressure
- 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting
- 2026Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures
- 2026ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems
- 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
- 2025SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
- 2025Deep Research Agents Brings Deeper Harm
- 2026An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios