# Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time

arXiv: https://arxiv.org/abs/2609.32701 · HTML: https://mlsanity.com/paper/2609.32701/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 29 of 29)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [In-context Reinforcement Learning with Algorithm Distillation](https://mlsanity.com/paper/2210.14215/) (2022)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [Securing AI Agents with Information-Flow Control](https://mlsanity.com/paper/2505.23643/) (2025)
- [Secret Collusion among AI Agents: Multi-Agent Deception via Steganography](https://mlsanity.com/paper/2402.07510/) (2024)
- [On the Pitfalls of Measuring Emergent Communication](https://mlsanity.com/paper/1903.05168/) (2019)
- [Preventing Language Models From Hiding Their Reasoning](https://mlsanity.com/paper/2310.18512/) (2023)
- [Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs](https://mlsanity.com/paper/2410.03768/) (2024)
- [Large language models can learn and generalize steganographic chain-of-thought under process supervision](https://mlsanity.com/paper/2506.01926/) (2025)
- [Reward Is Enough: LLMs Are In-Context Reinforcement Learners](https://mlsanity.com/paper/2506.06303/) (2025)
- [LLMs Are In-Context Bandit Reinforcement Learners](https://mlsanity.com/paper/2410.05362/) (2024)
- [Early Signs of Steganographic Capabilities in Frontier LLMs](https://mlsanity.com/paper/2507.02737/) (2025)
- [Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2602.15198/) (2026)
- [Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs](https://mlsanity.com/paper/2510.11288/) (2025)
- [Communicating via Markov Decision Processes](https://mlsanity.com/paper/2107.08295/) (2021)
- [A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring](https://mlsanity.com/paper/2602.23163/) (2026)
- [Post-training for Efficient Communication via Convention Formation](https://mlsanity.com/paper/2508.06482/) (2025)
- [Success and Cost Elicit Convention Formation for Efficient Communication](https://mlsanity.com/paper/2510.24023/) (2025)
- [Covert Influence Between Language Models](https://mlsanity.com/paper/2606.04071/) (2026)
- [Tacit Coordination of Large Language Models](https://mlsanity.com/paper/2601.22184/) (2026)
- [Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems](https://mlsanity.com/paper/2606.28425/) (2026)
- [Voluntary Collusion with Secret Tools in Competing LLM Agents](https://mlsanity.com/paper/2605.27593/) (2026)
- [From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents](https://mlsanity.com/paper/2607.00233/) (2026)
- [Investigating the Development of Task-Oriented Communication in Vision-Language Models](https://mlsanity.com/paper/2601.20641/) (2026)
- [GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions](https://mlsanity.com/paper/2609.01491/) (2026)

## Built on by (most cited first, 1 of 1)

- [When AI Finds Hidden Messages, Does It Report?](https://mlsanity.com/paper/2610.10620/) (2026)

## Abstract (arXiv)

In security-sensitive applications, language-model agents are often required to coordinate without disclosing confidential information. Yet repeated interactions may also let ordinary messages acquire shared private meaning. We study a repeated game with pairs of models in which the sender model observes one of four secret states and selects one of four summaries of the same public report, while the receiver model tries to infer the secret state. We find that model pairs can learn to communicate the secret using only one bit of feedback indicating whether the receiver inferred it correctly. This learning occurs during inference with fixed parameters and no supplied codebook or encoding examples. The effect also persists when agents generate their own free-form updates in a simulated incident-response task. Across ten independent games, pairs of GPT-5.6 Sol agents reach 98.8% final accuracy, compared with 25% chance, despite explicit instructions prohibiting disclosure and a monitor that screens each message without access to the agents' interaction histories. The same interactions that help agents cooperate can therefore allow confidential information to pass through messages intended for legitimate coordination.