mlsanityData to 14 Oct 2026

Despite Instructions: Frontier Agents Improvise Covert Channels at Test Time

arXiv 2609.32701 ↗First posted 26 Sep 2026cs.AI cs.CLcited by 1

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

In security-sensitive applications, language-model agents are often required to coordinate without disclosing confidential information. Yet repeated interactions may also let ordinary messages acquire shared private meaning. We study a repeated game with pairs of models in which the sender model observes one of four secret states and selects one of four summaries of the same public report, while the receiver model tries to infer the secret state. We find that model pairs can learn to communicate the secret using only one bit of feedback indicating whether the receiver inferred it correctly. This learning occurs during inference with fixed parameters and no supplied codebook or encoding examples. The effect also persists when agents generate their own free-form updates in a simulated incident-response task. Across ten independent games, pairs of GPT-5.6 Sol agents reach 98.8% final accuracy, compared with 25% chance, despite explicit instructions prohibiting disclosure and a monitor that screens each message without access to the agents' interaction histories. The same interactions that help agents cooperate can therefore allow confidential information to pass through messages intended for legitimate coordination.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 29 of its 42 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

29 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Tool Use Enables Undetectable Steganography in Multi-Agent LLM Systems25 Jun
  2. 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganography12 Feb

Other references · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  4. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  5. 2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
  6. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
  7. 2024Emergent social conventions and collective bias in LLM populationscited by 123
  8. 2025Securing AI Agents with Information-Flow Controlcited by 121
  9. 2019On the Pitfalls of Measuring Emergent Communicationcited by 71
  10. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  11. 2024Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMscited by 55
  12. 2025Large language models can learn and generalize steganographic chain-of-thought under process supervisioncited by 32
15 more
  1. 2025Reward Is Enough: LLMs Are In-Context Reinforcement Learnerscited by 31
  2. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 29
  3. 2025Early Signs of Steganographic Capabilities in Frontier LLMscited by 19
  4. 2026Colosseum: Auditing Collusion in Cooperative Multi-Agent Systemscited by 18
  5. 2025Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMscited by 17
  6. 2021Communicating via Markov Decision Processescited by 10
  7. 2026A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoringcited by 8
  8. 2025Post-training for Efficient Communication via Convention Formationcited by 7
  9. 2025Success and Cost Elicit Convention Formation for Efficient Communicationcited by 7
  10. 2026Covert Influence Between Language Modelscited by 6
  11. 2026Tacit Coordination of Large Language Modelscited by 5
  12. 2026Voluntary Collusion with Secret Tools in Competing LLM Agentscited by 4
  13. 2026From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agentscited by 4
  14. 2026Investigating the Development of Task-Oriented Communication in Vision-Language Modelscited by 2
  15. 2026GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactionscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When AI Finds Hidden Messages, Does It Report?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentSecBench: Measuring Prompt Injection, Privacy Leakage, and Tool-Use Integrity in LLM Agents
  2. 2026When Agents Lie: Premeditation, Persistence, and Exploitation in Repeated Games
  3. 2026Beyond the Transcript: Detecting Covert Co ordination in Latent Multi-Agent Communication
  4. 2025The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration
  5. 2026Undetectable Conversations Between AI Agents via Pseudorandom Noise-Resilient Key Exchange
  6. 2026LLMs Can't Play Hangman: On the Necessity of a Private Working Memory for Language Agents
  7. 2026Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations: A Black-Box, Cross-Principal Approach
  8. 2024Learn to Disguise: Avoid Refusal Responses in LLM's Defense via a Multi-agent Attacker-Disguiser Game