# VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization

arXiv: https://arxiv.org/abs/2503.04940 · HTML: https://mlsanity.com/paper/2503.04940/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 30 of 34)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020)
- [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017)
- [Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning](https://mlsanity.com/paper/2005.07064/) (2020)
- [Biases for Emergent Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1912.05676/) (2019)
- [How agents see things: On visual representations in an emergent language game](https://mlsanity.com/paper/1808.10696/) (2018)
- [EGG: a toolkit for research on Emergence of lanGuage in Games](https://mlsanity.com/paper/1907.00852/) (2019)
- [ShapeWorld - A new test methodology for multimodal language understanding](https://mlsanity.com/paper/1704.04517/) (2017)
- [Emergent Linguistic Phenomena in Multi-Agent Communication Games](https://mlsanity.com/paper/1901.08706/) (2019)
- [On the role of population heterogeneity in emergent communication](https://mlsanity.com/paper/2204.12982/) (2022)
- [Learning to Draw: Emergent Communication through Sketching](https://mlsanity.com/paper/2106.02067/) (2021)
- [Linking Emergent and Natural Languages via Corpus Transfer](https://mlsanity.com/paper/2203.13344/) (2022)
- [Word-order biases in deep-agent emergent communication](https://mlsanity.com/paper/1905.12330/) (2019)
- [Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence](https://mlsanity.com/paper/1611.03218/) (2016)
- [NWT: Towards natural audio-to-video generation with representation learning](https://mlsanity.com/paper/2106.04283/) (2021)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Emergent Language: A Survey and Taxonomy](https://mlsanity.com/paper/2409.02645/) (2024)

## Abstract (arXiv)

Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more closely mirror the discrete nature of human language, learning such protocols remains fundamentally difficult due to the non-differentiability of symbol sampling. Existing approaches typically rely on high-variance gradient estimators such as REINFORCE or on continuous relaxations such as Gumbel-Softmax, both of which suffer from limitations in training stability and scalability. Motivated by cognitive theories that emphasize intrapersonal processes preceding communication, we explore self-play as a substrate for language emergence prior to mutual interaction. We introduce Vector Quantized Emergent Language (VQEL), a novel architecture that incorporates vector quantization into the message generation process. VQEL enables agents to perform self-play using discrete internal representations derived from a learned codebook while preserving end-to-end differentiability. Moreover, the resulting vector-quantized codebook naturally induces a symbolic vocabulary that can be directly transferred and aligned during subsequent mutual play with other agents. Empirical results show that agents pretrained via VQEL self-play achieve more consistent symbol alignment and higher task success when later engaged in mutual interaction. These findings position self-play as a principled and effective mechanism for learning discrete communication protocols, addressing key optimization and representational challenges in emergent language systems.