# Learning Multi-Object Positional Relationships via Emergent Communication arXiv: https://arxiv.org/abs/2302.08084 ยท HTML: https://mlsanity.com/paper/2302.08084/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 40 references that are in the map. - First posted on arXiv 16 Feb 2023. ## Builds on (most cited first, 30 of 31) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020) - [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017) - [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016) - [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017) - [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020) - [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018) - [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017) - [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020) - [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017) - [Ease-of-Teaching and Language Structure from Emergent Communication](https://mlsanity.com/paper/1906.02403/) (2019) - [Anti-efficient encoding in emergent communication](https://mlsanity.com/paper/1905.12561/) (2019) - [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019) - [Environmental drivers of systematicity and generalization in a situated agent](https://mlsanity.com/paper/1910.00571/) (2019) - [Biases for Emergent Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1912.05676/) (2019) - [How agents see things: On visual representations in an emergent language game](https://mlsanity.com/paper/1808.10696/) (2018) - [Towards Deep Symbolic Reinforcement Learning](https://mlsanity.com/paper/1609.05518/) (2016) - [Learning to Ground Multi-Agent Communication with Autoencoders](https://mlsanity.com/paper/2110.15349/) (2021) - [Compositional Obverter Communication Learning From Raw Visual Input](https://mlsanity.com/paper/1804.02341/) (2018) - [The Emergence of Compositional Languages for Numeric Concepts Through Iterated Learning in Neural Agents](https://mlsanity.com/paper/1910.05291/) (2019) - [Interpretable agent communication from scratch (with a generic visual processor emerging on the side)](https://mlsanity.com/paper/2106.04258/) (2021) - [Emergent Discrete Communication in Semantic Spaces](https://mlsanity.com/paper/2108.01828/) (2021) - [Emergence of Communication in an Interactive World with Consistent Speakers](https://mlsanity.com/paper/1809.00549/) (2018) - [Co-evolution of language and agents in referential games](https://mlsanity.com/paper/2001.03361/) (2020) - [ReferentialGym: A Nomenclature and Framework for Language Emergence & Grounding in (Visual) Referential Games](https://mlsanity.com/paper/2012.09486/) (2020) - [On (Emergent) Systematic Generalisation and Compositionality in Visual Referential Games with Straight-Through Gumbel-Softmax Estimator](https://mlsanity.com/paper/2012.10776/) (2020) - [Dynamic population-based meta-learning for multi-agent communication with natural language](https://mlsanity.com/paper/2110.14241/) (2021) - [Towards Multi-Agent Communication-Based Language Learning](https://mlsanity.com/paper/1605.07133/) (2016) - [Avoiding hashing and encouraging visual semantics in referential emergent language games](https://mlsanity.com/paper/1911.05546/) (2019) ## Built on by (most cited first, 4 of 4) - [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024) - [Emergent Language: A Survey and Taxonomy](https://mlsanity.com/paper/2409.02645/) (2024) - [Cognitively-Inspired Emergent Communication via Knowledge Graphs for Assisting the Visually Impaired](https://mlsanity.com/paper/2505.22087/) (2025) - [Automated Semantic Rules Detection (ASRD) for Emergent Communication Interpretation](https://mlsanity.com/paper/2601.03254/) (2026) ## Abstract (arXiv) The study of emergent communication has been dedicated to interactive artificial intelligence. While existing work focuses on communication about single objects or complex image scenes, we argue that communicating relationships between multiple objects is important in more realistic tasks, but understudied. In this paper, we try to fill this gap and focus on emergent communication about positional relationships between two objects. We train agents in the referential game where observations contain two objects, and find that generalization is the major problem when the positional relationship is involved. The key factor affecting the generalization ability of the emergent language is the input variation between Speaker and Listener, which is realized by a random image generator in our work. Further, we find that the learned language can generalize well in a new multi-step MDP task where the positional relationship describes the goal, and performs better than raw-pixel images as well as pre-trained image features, verifying the strong generalization ability of discrete sequences. We also show that language transfer from the referential game performs better in the new task than learning language directly in this task, implying the potential benefits of pre-training in referential games. All in all, our experiments demonstrate the viability and merit of having agents learn to communicate positional relationships between multiple objects through emergent communication.