mlsanityData to 14 Oct 2026

Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments

arXiv 2311.04453 ↗First posted 8 Nov 2023cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

As a sub-discipline of evolutionary and computational linguistics, emergent communication (EC) studies communication protocols, called emergent languages, arising in simulations where agents communicate. A key goal of EC is to give rise to languages that share statistical properties with natural languages. In this paper, we reinterpret Lewis's signaling game, a frequently used setting in EC, as beta-VAE and reformulate its objective function as ELBO. Consequently, we clarify the existence of prior distributions of emergent languages and show that the choice of the priors can influence their statistical properties. Specifically, we address the properties of word lengths and segmentation, known as Zipf's law of abbreviation (ZLA) and Harris's articulation scheme (HAS), respectively. It has been reported that the emergent languages do not follow them when using the conventional objective. We experimentally demonstrate that by selecting an appropriate prior distribution, more natural segments emerge, while suggesting that the conventional one prevents the languages from following ZLA and HAS.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 51 of its 91 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

51 of 91 references.

References · 51

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2016Layer Normalizationcited by 5,669
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  6. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  8. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  9. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  10. 2016Deep Variational Information Bottleneckcited by 930
  11. 2015Generating Sentences from a Continuous Spacecited by 612
  12. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
38 more
  1. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  2. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  3. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  4. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 185
  5. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 170
  6. 2015A Theoretically Grounded Application of Dropout in Recurrent Neural Networkscited by 161
  7. 2016Discrete Variational Autoencoderscited by 125
  8. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 125
  9. 2015Gradient Estimation Using Stochastic Computation Graphscited by 121
  10. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 119
  11. 2014Neural Variational Inference and Learning in Belief Networkscited by 117
  12. 2016Recurrent Neural Network Grammarscited by 109
  13. 2020Compositionality and Generalization in Emergent Languagescited by 103
  14. 2018Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Inputcited by 102
  15. 2018Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networkscited by 101
  16. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  17. 2019Measuring Compositionality in Representation Learningcited by 84
  18. 2020Compositional Languages Emerge in a Neural Iterated Learning Modelcited by 75
  19. 2020Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learningcited by 66
  20. 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
  21. 2019Anti-efficient encoding in emergent communicationcited by 59
  22. 2019Learning Hierarchical Priors in VAEscited by 53
  23. 2022A survey of multimodal deep generative modelscited by 47
  24. 2022Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Languagecited by 42
  25. 2022Context Limitations Make Neural Language Models More Human-Likecited by 34
  26. 2018DVAE++: Discrete Variational Autoencoders with Overlapping Transformationscited by 32
  27. 2018Compositional Obverter Communication Learning From Raw Visual Inputcited by 30
  28. 2019Capacity, Bandwidth, and Compositionality in Emergent Language Learningcited by 30
  29. 2022Emergent Communication: Generalization and Overfitting in Lewis Gamescited by 28
  30. 2021Interpretable agent communication from scratch (with a generic visual processor emerging on the side)cited by 25
  31. 2022On the role of population heterogeneity in emergent communicationcited by 24
  32. 2020"LazImpa": Lazy and Impatient neural agents learn to communicate efficientlycited by 23
  33. 2022Emergent Communication for Understanding Human Language Evolution: What's Missing?cited by 23
  34. 2018DVAE#: Discrete Variational Autoencoders with Relaxed Boltzmann Priorscited by 20
  35. 2022Emergent Communication through Metropolis-Hastings Naming Game with Deep Generative Modelscited by 16
  36. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 13
  37. 2020The Grammar of Emergent Languagescited by 9
  38. 2023Recursive Metropolis-Hastings Naming Game: Symbol Emergence in a Multi-agent System based on Probabilistic Generative Modelscited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 8; 2026: 2.

11 papers.

  1. 2024Generative Emergent Communication: Large Language Model is a Collective World Modelcited by 15
  2. 2026Variable-Length Semantic IDs for Recommender Systemscited by 4
  3. 2025Reward-Independent Messaging for Decentralized Multi-Agent Reinforcement Learningcited by 2
  4. 2025A Compressive-Expressive Communication Framework for Compositional Representationscited by 1
  5. 2025From Grunts to Lexicons: Emergent Language from Cooperative Foragingcited by 1
  6. 2025Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Datacited by 1
  7. 2025Cognitively-Inspired Emergent Communication via Knowledge Graphs for Assisting the Visually Impairedcited by 0
  8. 2025Co-Creative Learning via Metropolis-Hastings Interaction between Humans and AIcited by 0
  9. 2025Why Are Parsing Actions for Understanding Message Hierarchies Not Random?cited by 0
  10. 2025Discovering Properties of Inflectional Morphology in Neural Emergent Communicationcited by 0
  11. 2026Translating Signals to Languages for sEMG-Based Activity Recognitioncited by 0