# Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

arXiv: https://arxiv.org/abs/2606.08081 · HTML: https://mlsanity.com/paper/2606.08081/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 6 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)
- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 17 of 17)

- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [A Survey on Large Language Model-Based Game Agents](https://mlsanity.com/paper/2404.02039/) (2024)
- [Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents](https://mlsanity.com/paper/2402.12327/) (2024)
- [Grounding Gaps in Language Model Generations](https://mlsanity.com/paper/2311.09144/) (2023)
- [Characterizing the dynamics of learning in repeated reference games](https://mlsanity.com/paper/1912.07199/) (2019)
- [From partners to populations: A hierarchical Bayesian account of coordination and convention](https://mlsanity.com/paper/2104.05857/) (2021)
- [Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs](https://mlsanity.com/paper/2408.01417/) (2024)
- [Measuring How (Not Just Whether) VLMs Build Common Ground](https://mlsanity.com/paper/2509.03805/) (2025)
- [LVLMs are Bad at Overhearing Human Referential Communication](https://mlsanity.com/paper/2509.11514/) (2025)
- [LVLMs and Humans Ground Differently in Referential Communication](https://mlsanity.com/paper/2601.19792/) (2026)
- [Towards Multi-Agent Communication-Based Language Learning](https://mlsanity.com/paper/1605.07133/) (2016)
- [Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes](https://mlsanity.com/paper/2509.05882/) (2025)
- [Success and Cost Elicit Convention Formation for Efficient Communication](https://mlsanity.com/paper/2510.24023/) (2025)
- [Evaluating Behavioral Alignment in Conflict Dialogue: A Multi-Dimensional Comparison of LLM Agents and Humans](https://mlsanity.com/paper/2509.16394/) (2025)
- [Analysing Cross-Speaker Convergence in Face-to-Face Dialogue through the Lens of Automatically Detected Shared Linguistic Constructions](https://mlsanity.com/paper/2405.08546/) (2024)
- [LEEETs-Dial: Linguistic Entrainment in End-to-End Task-oriented Dialogue systems](https://mlsanity.com/paper/2311.09390/) (2023)
- [Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data](https://mlsanity.com/paper/2507.08610/) (2025)

## Abstract (arXiv)

Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific expressions grounded in shared interaction history; that is, with conceptual pacts. Prior work shows that multimodal LLMs fail to become more efficient across rounds, although they align on the labels they use. However, how can we determine whether this alignment reflects partner-specific grounding rather than a shared task vocabulary? We address this by comparing competent multimodal agent dyads with human dyads from the KTH Tangrams corpus. Our novel methodological contribution is a pragmatically constrained pseudo-dyad baseline: rounds from two different real dyads describing the same target at comparable trajectory positions are paired, preserving referential task structure while removing shared partner history. This enables us to test whether the observed label alignment depends on interaction with a specific partner. Across three measures (task competence, description strategy, alignment dynamics), we find clear differences. Humans reduce effort through entrainment, compressing descriptions and increasing label alignment with partners. Agents instead maintain fixed effort levels, producing verbose descriptions from round one, with near-ceiling label overlap that is statistically indistinguishable between real and pseudo dyads. MLLMs thus achieve coordination without conceptual pacts, succeeding by verbose description rather than by forming the compact, history-dependent referring expressions characteristic of human dialogue.