# Sanzaru: Benchmarking Protocol Formation under Epistemic & Communication Asymmetry

arXiv: https://arxiv.org/abs/2610.15163 · HTML: https://mlsanity.com/paper/2610.15163/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 10 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits](https://mlsanity.com/paper/2305.02547/) (2023)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023)
- [Emergent social conventions and collective bias in LLM populations](https://mlsanity.com/paper/2410.08948/) (2024)
- [PersonaGym: Evaluating Persona Agents and LLMs](https://mlsanity.com/paper/2407.18416/) (2024)
- [On the Pitfalls of Measuring Emergent Communication](https://mlsanity.com/paper/1903.05168/) (2019)
- [LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models](https://mlsanity.com/paper/2310.03903/) (2023)
- [SimpleToM: Exposing the Gap between Explicit ToM Inference and Implicit ToM Application in LLMs](https://mlsanity.com/paper/2410.13648/) (2024)
- [Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents](https://mlsanity.com/paper/2502.20073/) (2025)
- [COMMA: A Communicative Multimodal Multi-Agent Benchmark](https://mlsanity.com/paper/2410.07553/) (2024)
- [Benchmarking LLMs' Swarm intelligence](https://mlsanity.com/paper/2505.04364/) (2025)
- [PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraints](https://mlsanity.com/paper/2604.11523/) (2026)
- [Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models](https://mlsanity.com/paper/2406.14035/) (2024)
- [Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry](https://mlsanity.com/paper/2510.25595/) (2025)
- [SOTOPIA-TOM: Evaluating Privacy and Information Management in Multi-Agent Interaction with Theory of Mind](https://mlsanity.com/paper/2605.02307/) (2026)
- [LLM Agents for Deliberative Collaboration: A Study on Joint Decision Making Under Partial Observability](https://mlsanity.com/paper/2607.06157/) (2026)
- [GPTNT: Benchmarking Real-Time Collaboration Between Multimodal Agents on Keep Talking And Nobody Explodes](https://mlsanity.com/paper/2606.28514/) (2026)

## Abstract (arXiv)

A language model's ability to solve a task individually does not ensure that a team can solve it under unequal access to information and communication. SANZARU examines this gap through cooperative bomb-defusal puzzles adapted from BOMBANANA! (Lefto Studio, 2026). Task information and action authority are divided among three agents: one cannot hear, another cannot speak, and the third cannot see. The benchmark contains 120 puzzle instances across seven families, each targeting a distinct coordination ability. Across 1512 episodes, 7 models achieve an overall success rate of 49%, ranging from 20% to 86% under the evaluated configurations. Teams develop or adopt diverse communication protocols, from simple gesture codes to Morse and encodings of entire manuals. Yet trace analysis attributes about 60% of 771 team losses to communication, and agents attempt to use channels unavailable to their teammates despite explicit role instructions. Together, task outcomes and interaction traces enable evaluation of both collaborative performance and perspective taking throughout cooperation. The benchmark is available at https://github.com/markli404/sanzaru-benchmark.