mlsanityData to 7 Oct 2026

Compositional Languages Emerge in a Neural Iterated Learning Model

arXiv 2002.01365 ↗First posted 4 Feb 2020cs.CL cs.AI cs.LGcited by 75

Not on the map. Why?

Abstract (arXiv)

The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalizing power of the neural agent communication.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 9; 2023: 17; 2024: 20; 2025: 5; 2026: 11.

75 papers.

Sort:

By area:

  1. 2022The Primacy Bias in Deep Reinforcement Learningcited by 242
  2. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 97
  3. 2024Learning Dynamics of LLM Finetuningcited by 93
  4. 2023Break It Down: Evidence for Structural Compositionality in Neural Networkscited by 44
  5. 2022Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Languagecited by 42
  6. 2021Emergent Communication of Generalizationscited by 41
  7. 2022Fortuitous Forgetting in Connectionist Networkscited by 39
  8. 2022Emergent Communication: Generalization and Overfitting in Lewis Gamescited by 28
  9. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  10. 2021Learning to Draw: Emergent Communication through Sketchingcited by 23
  11. 2021Iterated learning for emergent systematicity in VQAcited by 21
  12. 2021Language Modeling, Lexical Translation, Reordering: The Training Process of NMT through the Lens of Classical SMTcited by 21
38 more
  1. 2021A comparative evaluation and analysis of three generations of Distributional Semantic Modelscited by 19
  2. 2021Multi-label Iterated Learning for Image Classification with Label Ambiguitycited by 19
  3. 2024Iterated Learning Improves Compositionality in Large Vision-Language Modelscited by 17
  4. 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspectivecited by 17
  5. 2024A Review of the Applications of Deep Learning-Based Emergent Communicationcited by 17
  6. 2021Emergent Graphical Conventions in a Visual Communication Gamecited by 16
  7. 2023Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddingscited by 15
  8. 2024Generative Emergent Communication: Large Language Model is a Collective World Modelcited by 15
  9. 2024On The Specialization of Neural Modulescited by 14
  10. 2021The Emergence of the Shape Bias Results from Communicative Efficiencycited by 13
  11. 2023What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structurecited by 13
  12. 2024A Complexity-Based Theory of Compositionalitycited by 13
  13. 2025Does Data Scaling Lead to Visual Compositional Generalization?cited by 13
  14. 2021Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communicationcited by 12
  15. 2023Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-offcited by 12
  16. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  17. 2021Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictabilitycited by 11
  18. 2023Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segmentscited by 11
  19. 2023Referential communication in heterogeneous communities of pre-trained visual deep networkscited by 10
  20. 2023Cultural evolution via iterated learning and communication explains efficient color naming systemscited by 10
  21. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  22. 2024The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communicationcited by 7
  23. 2024lpNTK: Better Generalisation with Less Data via Sample Interaction During Learningcited by 6
  24. 2026Generative AI collective behavior needs an interactionist paradigmcited by 6
  25. 2023Visual Referential Games Further the Emergence of Disentangled Representationscited by 5
  26. 2023Metropolis-Hastings algorithm in joint-attention naming game: Experimental semiotics studycited by 5
  27. 2024Language Evolution with Deep Learningcited by 5
  28. 2024Understanding Simplicity Bias towards Compositional Mappings via Learning Dynamicscited by 5
  29. 2023Learning Multi-Object Positional Relationships via Emergent Communicationcited by 4
  30. 2024Long Tail Image Generation Through Feature Space Augmentation and Iterated Learningcited by 4
  31. 2026From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agentscited by 4
  32. 2021TexRel: a Green Family of Datasets for Emergent Communications on Relationscited by 3
  33. 2024An iterated learning model of language change that mixes supervised and unsupervised learningcited by 3
  34. 2021Zero-Shot Generalization using Intrinsically Motivated Compositional Emergent Protocolscited by 2
  35. 2022Meta-Referential Games to Learn Compositional Learning Behaviourscited by 2
  36. 2023Models of symbol emergence in communication: a conceptual review and a guide for avoiding local minimacited by 2
  37. 2023Emergent Communication with Attentioncited by 2
  38. 2024"Forgetting" in Machine Learning and Beyond: A Surveycited by 2

50 most cited of 75 All 75 (JSON)