mlsanityData to 14 Oct 2026

For Your Eyes Only: Evaluating Coordination Between Isolated Language Model Instances

arXiv 2609.19504 ↗First posted 16 Sep 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

As model-generated content is increasingly consumed by other model instances in automated workflows, a practically important question arises: can a model embed a signal in natural language that an independent instance of the same model can detect, relying only on shared pre-training and task instructions, without any shared memory or coordination-specific training? We introduce For Your Eyes Only, a cooperative signalling game designed to evaluate this directly. A Sender produces free-form descriptions for two words, one of which is a hidden target; an isolated Receiver must identify it. We evaluate seven contemporary models from four architectural families on 300 word pairs from established psycholinguistic corpora, using the Double-Pass Success Rate to control for output biases. We find that most models struggle to maintain coordination once they are required to avoid detectable signals, while one frontier model retains near-perfect performance even after such filtering. We further show that models can direct this capability toward deliberate misdirection, and that coordination is consistently weaker across architectures than within them.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

21 of 36 references.

References · 21

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  4. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  5. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  6. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  7. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  8. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  9. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
  10. 2014SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimationcited by 172
  11. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
  12. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
9 more
  1. 2019Neural Linguistic Steganographycited by 78
  2. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  3. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 49
  4. 2025Beyond the Surface: Measuring Self-Preference in LLM Judgmentscited by 31
  5. 2025The Steganographic Potentials of Language Modelscited by 10
  6. 2025TrojanStego: Your Language Model Can Secretly Be A Steganographic Privacy Leaking Agentcited by 9
  7. 2025The Price of Thought: A Multilingual Analysis of Reasoning, Performance, and Cost of Negotiation in Large Language Modelscited by 5
  8. 2025clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisationscited by 4
  9. 2025From Templates to Natural Language: Generalization Challenges in Instruction-Tuned LLMs for Spatial Reasoningcited by 1

Built on by

Not cited yet.