mlsanityData to 7 Oct 2026

Concept-Best-Matching: Evaluating Compositionality in Emergent Communication

arXiv 2403.14705 ↗First posted 17 Mar 2024cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Artificial agents that learn to communicate in order to accomplish a given task acquire communication protocols that are typically opaque to a human. A large body of work has attempted to evaluate the emergent communication via various evaluation measures, with compositionality featuring as a prominent desired trait. However, current evaluation procedures do not directly expose the compositionality of the emergent communication. We propose a procedure to assess the compositionality of emergent communication by finding the best-match between emerged words and natural language concepts. The best-match algorithm provides both a global score and a translation-map from emergent words to natural language concepts. To the best of our knowledge, it is the first time that such direct and interpretable mapping between emergent words and human concepts is provided.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 17 Mar 2024.

Builds on

14 of 24 references.

References · 14

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  2. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  3. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  4. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 123
  5. 2020Compositionality and Generalization in Emergent Languagescited by 103
  6. 2018Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Inputcited by 102
  7. 2016Reasoning About Pragmatics with Neural Listeners and Speakerscited by 68
  8. 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
  9. 2019EGG: a toolkit for research on Emergence of lanGuage in Gamescited by 42
  10. 2021Emergent Communication of Generalizationscited by 41
  11. 2017ShapeWorld - A new test methodology for multimodal language understandingcited by 37
  12. 2021Iterated learning for emergent systematicity in VQAcited by 21
2 more
  1. 2020A Rate-Distortion view of human pragmatic reasoningcited by 19
  2. 2022Linking Emergent and Natural Languages via Corpus Transfercited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 3.

7 papers.

  1. 2024Emergent Language: A Survey and Taxonomycited by 12
  2. 2025Unsupervised Translation of Emergent Communicationcited by 3
  3. 2024Semantics and Spatiality of Emergent Communicationcited by 2
  4. 2024SimSiam Naming Game: A Unified Approach for Emergent Communication and Representation Learningcited by 1
  5. 2026CtD: Composition through Decomposition in Emergent Communicationcited by 1
  6. 2026Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketchingcited by 0
  7. 2026EmCom-Diffusion: Probing Visual Reflection in Emergent Languages via Image Generationcited by 0