# The Convention Gap: Beyond Literal Content in Human-AI Cooperation

arXiv: https://arxiv.org/abs/2609.11489 · HTML: https://mlsanity.com/paper/2609.11489/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Builds on (most cited first, 14 of 14)

- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Off-Belief Learning](https://mlsanity.com/paper/2103.04000/) (2021)
- [Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.01458/) (2018)
- [Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1912.02288/) (2019)
- [Evaluation of Human-AI Teams for Learned and Rule-Based Agents in Hanabi](https://mlsanity.com/paper/2107.07630/) (2021)
- [Any-Play: An Intrinsic Augmentation for Zero-Shot Coordination](https://mlsanity.com/paper/2201.12436/) (2022)
- [Ad-Hoc Human-AI Coordination Challenge](https://mlsanity.com/paper/2506.21490/) (2025)
- [Information Theory of Meaningful Communication](https://mlsanity.com/paper/2411.12728/) (2024)
- [Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents](https://mlsanity.com/paper/2601.18077/) (2026)
- [Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi](https://mlsanity.com/paper/2412.06333/) (2024)
- [Behavioral Differences is the Key of Ad-hoc Team Cooperation in Multiplayer Games Hanabi](https://mlsanity.com/paper/2303.06775/) (2023)
- [Reinforcement Learning for Hanabi](https://mlsanity.com/paper/2506.00458/) (2025)

## Built on by (most cited first, 1 of 1)

- [What and Where in Identifying and Quantifying Conventions in Humans and AI Agents](https://mlsanity.com/paper/2610.20205/) (2026)

## Abstract (arXiv)

Cooperative AI agents are often evaluated against other AIs, yet human cooperation is thought to rely on implicit conventions: shared rules for reading meaning beyond the literal message. We propose the convention gap as a diagnostic of such implicit communication. In the card game Hanabi, where hints are the only messages about one's own hand, the convention gap is the failure probability predicted exactly from the literal content of the hints and the visible cards, minus the observed failure rate of the cards played. We prove that the population gap (the value the gap approaches over many games) is zero when the partner's choices convey nothing beyond their literal content, given a fair shuffle and a player who uses only what was said and seen. A nonzero population gap thus indicates that partner choices conveyed more; it does not measure how much. The gap was +26.2 percentage points (pp) in public human-human games (+46 pp on plays of cards that no hint had pointed to), +16.4 pp for human plays with AI partners, and -0.6 pp in AI-AI games we simulated among rule-based agents. With three AI agents of a published human-AI study, the literal failure probability of the cards humans played was similar across partners (38-41%), yet human failure ranged from 14.4% to 34.4% and the gap from +24.1 to +6.2 pp, and the contrast between partners persisted within participants. Across five levels of Off-Belief Learning agents, trained so that the first level reads past actions only for their literal content and each higher level interprets more, the gap rose monotonically from +1.6 to +21.7 pp. These results argue for reporting the convention gap on human-partnered play, not only AI-AI performance, when evaluating agents intended for human partners.