mlsanityData to 14 Oct 2026

Language Evolution with Deep Learning

arXiv 2403.11958 ↗First posted 18 Mar 2024cs.CL cs.MAcited by 5

Not on the map. Why?

Abstract (arXiv)

Computational modeling plays an essential role in the study of language emergence. It aims to simulate the conditions and learning processes that could trigger the emergence of a structured language within a simulated controlled environment. Several methods have been used to investigate the origin of our language, including agent-based systems, Bayesian agents, genetic algorithms, and rule-based systems. This chapter explores another class of computational models that have recently revolutionized the field of machine learning: deep learning models. The chapter introduces the basic concepts of deep and reinforcement learning methods and summarizes their helpfulness for simulating language emergence. It also discusses the key findings, limitations, and recent attempts to build realistic simulations. This chapter targets linguists and cognitive scientists seeking an introduction to deep learning as a tool to investigate language evolution.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 52 of its 140 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

52 of 140 references.

References · 52

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  10. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  11. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  12. 2016Layer Normalizationcited by 5,669
38 more
  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  7. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  9. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  10. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  11. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  12. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  13. 2018DropBlock: A regularization method for convolutional networkscited by 262
  14. 2018TarMAC: Targeted Multi-Agent Communicationcited by 208
  15. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
  16. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
  17. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 125
  18. 2020Compositionality and Generalization in Emergent Languagescited by 103
  19. 2018Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Inputcited by 102
  20. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  21. 2019Measuring Compositionality in Representation Learningcited by 84
  22. 2020Compositional Languages Emerge in a Neural Iterated Learning Modelcited by 75
  23. 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
  24. 2019Anti-efficient encoding in emergent communicationcited by 59
  25. 2020Countering Language Drift with Seeded Iterated Learningcited by 57
  26. 2018How agents see things: On visual representations in an emergent language gamecited by 45
  27. 2019EGG: a toolkit for research on Emergence of lanGuage in Gamescited by 42
  28. 2019Emergent Linguistic Phenomena in Multi-Agent Communication Gamescited by 36
  29. 2019Capacity, Bandwidth, and Compositionality in Emergent Language Learningcited by 30
  30. 2022Emergent Communication: Generalization and Overfitting in Lewis Gamescited by 28
  31. 2021Interpretable agent communication from scratch (with a generic visual processor emerging on the side)cited by 25
  32. 2022On the role of population heterogeneity in emergent communicationcited by 24
  33. 2020"LazImpa": Lazy and Impatient neural agents learn to communicate efficientlycited by 23
  34. 2022Emergent Communication for Understanding Human Language Evolution: What's Missing?cited by 23
  35. 2022Linking Emergent and Natural Languages via Corpus Transfercited by 19
  36. 2020Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populationscited by 18
  37. 2019Emergence of Compositional Language with Deep Generational Transmissioncited by 17
  38. 2019Shaping representations through communication: community size effect in artificial learning systemscited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communicationcited by 7
  2. 2025Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Modelscited by 4
  3. 2024SimSiam Naming Game: A Unified Approach for Emergent Communication and Representation Learningcited by 1
  4. 2026A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Datacited by 0
  5. 2026TRACE: Transparent Retrieval for Abstract Concept Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Emergent Language: A Survey and Taxonomy
  2. 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspective
  3. 2024A Review of the Applications of Deep Learning-Based Emergent Communication
  4. 2024Searching for Structure: Investigating Emergent Communication with Large Language Models
  5. 2024Opening the black box of language acquisition
  6. 2025The Emergence of Grammar through Reinforcement Learning
  7. 2025Deep Research: A Systematic Survey
  8. 2026GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions