# Discrete-Valued Neural Communication

arXiv: https://arxiv.org/abs/2107.02367 · HTML: https://mlsanity.com/paper/2107.02367/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 59 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Neural Module Networks](https://mlsanity.com/paper/1511.02799/) (2015)
- [PathNet: Evolution Channels Gradient Descent in Super Neural Networks](https://mlsanity.com/paper/1701.08734/) (2017)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Contrastive Learning of Structured World Models](https://mlsanity.com/paper/1911.12247/) (2019)
- [Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning](https://mlsanity.com/paper/1711.01239/) (2017)
- [Robust Large Margin Deep Neural Networks](https://mlsanity.com/paper/1605.08254/) (2016)
- [Entity Abstraction in Visual Model-Based Reinforcement Learning](https://mlsanity.com/paper/1910.12827/) (2019)
- [Neural Programmer-Interpreters](https://mlsanity.com/paper/1511.06279/) (2015)
- [Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactions](https://mlsanity.com/paper/1802.10353/) (2018)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [Coordination Among Neural Modules Through a Shared Global Workspace](https://mlsanity.com/paper/2103.01197/) (2021)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations](https://mlsanity.com/paper/1606.01305/) (2016)
- [Routing Networks and the Challenges of Modular and Compositional Computation](https://mlsanity.com/paper/1904.12774/) (2019)
- [Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems](https://mlsanity.com/paper/2006.16225/) (2020)
- [Generalization Error of Invariant Classifiers](https://mlsanity.com/paper/1610.04574/) (2016)
- [Conditional Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/2106.11083/) (2021)
- [Relational Forward Models for Multi-Agent Learning](https://mlsanity.com/paper/1809.11044/) (2018)
- [Transformers with Competitive Ensembles of Independent Mechanisms](https://mlsanity.com/paper/2103.00336/) (2021)
- [Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication](https://mlsanity.com/paper/1901.03887/) (2019)

## Built on by (most cited first, 30 of 39)

- [Universal Humanoid Motion Representations for Physics-Based Control](https://mlsanity.com/paper/2310.04582/) (2023)
- [How Does Information Bottleneck Help Deep Learning?](https://mlsanity.com/paper/2305.18887/) (2023)
- [Trustworthy Graph Neural Networks: Aspects, Methods and Trends](https://mlsanity.com/paper/2205.07424/) (2022)
- [Disentanglement via Latent Quantization](https://mlsanity.com/paper/2305.18378/) (2023)
- [Neural Systematic Binder](https://mlsanity.com/paper/2211.01177/) (2022)
- [From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence](https://mlsanity.com/paper/2110.15245/) (2021)
- [The Relational Bottleneck as an Inductive Bias for Efficient Abstraction](https://mlsanity.com/paper/2309.06629/) (2023)
- [Simplicial Embeddings in Self-Supervised Learning and Downstream Classification](https://mlsanity.com/paper/2204.00616/) (2022)
- [Robustness Implies Generalization via Data-Dependent Generalization Bounds](https://mlsanity.com/paper/2206.13497/) (2022)
- [Learning Invariant Molecular Representation in Latent Discrete Space](https://mlsanity.com/paper/2310.14170/) (2023)
- [Discrete Key-Value Bottleneck](https://mlsanity.com/paper/2207.11240/) (2022)
- [Improving Compositional Generalization Using Iterated Learning and Simplicial Embeddings](https://mlsanity.com/paper/2310.18777/) (2023)
- [Representation Learning in Deep RL via Discrete Information Bottleneck](https://mlsanity.com/paper/2212.13835/) (2022)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantization](https://mlsanity.com/paper/2202.01334/) (2022)
- [Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning](https://mlsanity.com/paper/2211.00247/) (2022)
- [Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.03022/) (2022)
- [Unlearning via Sparse Representations](https://mlsanity.com/paper/2311.15268/) (2023)
- [Towards Principled Representation Learning from Videos for Reinforcement Learning](https://mlsanity.com/paper/2403.13765/) (2024)
- [Mitigating Adversarial Perturbations for Deep Reinforcement Learning via Vector Quantization](https://mlsanity.com/paper/2410.03376/) (2024)
- [Improving Discrete Optimisation Via Decoupled Straight-Through Estimator](https://mlsanity.com/paper/2410.13331/) (2024)
- [Neurosymbolic Systems of Perception & Cognition: The Role of Attention](https://mlsanity.com/paper/2112.01603/) (2021)
- [Reusable Slotwise Mechanisms](https://mlsanity.com/paper/2302.10503/) (2023)
- [Balance of Number of Embedding and their Dimensions in Vector Quantization](https://mlsanity.com/paper/2407.04939/) (2024)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [Towards Equipping Transformer with the Ability of Systematic Compositionality](https://mlsanity.com/paper/2312.07280/) (2023)
- [Symbolic Autoencoding for Self-Supervised Sequence Learning](https://mlsanity.com/paper/2402.10575/) (2024)
- [VQSynery: Robust Drug Synergy Prediction With Vector Quantization Mechanism](https://mlsanity.com/paper/2403.03089/) (2024)
- [Gaussian Mixture Vector Quantization with Aggregated Categorical Posterior](https://mlsanity.com/paper/2410.10180/) (2024)
- [Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents](https://mlsanity.com/paper/2312.15985/) (2023)

## Abstract (arXiv)

Deep learning has advanced from fully connected architectures to structured models organized into components, e.g., the transformer composed of positional elements, modular architectures divided into slots, and graph neural nets made up of nodes. In structured models, an interesting question is how to conduct dynamic and possibly sparse communication among the separate components. Here, we explore the hypothesis that restricting the transmitted information among components to discrete representations is a beneficial bottleneck. The motivating intuition is human language in which communication occurs through discrete symbols. Even though individuals have different understandings of what a "cat" is based on their specific experiences, the shared discrete token makes it possible for communication among individuals to be unimpeded by individual differences in internal representation. To discretize the values of concepts dynamically communicated among specialist components, we extend the quantization mechanism from the Vector-Quantized Variational Autoencoder to multi-headed discretization with shared codebooks and use it for discrete-valued neural communication (DVNC). Our experiments show that DVNC substantially improves systematic generalization in a variety of architectures -- transformers, modular architectures, and graph neural networks. We also show that the DVNC is robust to the choice of hyperparameters, making the method very useful in practice. Moreover, we establish a theoretical justification of our discretization process, proving that it has the ability to increase noise robustness and reduce the underlying dimensionality of the model.