# Neurosymbolic Transformers for Multi-Agent Communication

arXiv: https://arxiv.org/abs/2101.03238 · HTML: https://mlsanity.com/paper/2101.03238/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 35 references that are in the map.
- First posted on arXiv 5 Jan 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Programmatically Interpretable Reinforcement Learning](https://mlsanity.com/paper/1804.02477/) (2018)
- [Verifiable Reinforcement Learning via Policy Extraction](https://mlsanity.com/paper/1805.08328/) (2018)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [Learning to Infer Graphics Programs from Hand-Drawn Images](https://mlsanity.com/paper/1707.09627/) (2017)
- [Learning Decentralized Controllers for Robot Swarms with Graph Neural Networks](https://mlsanity.com/paper/1903.10527/) (2019)
- [Falling Rule Lists](https://mlsanity.com/paper/1411.5899/) (2014)
- [HOUDINI: Lifelong Learning as Program Synthesis](https://mlsanity.com/paper/1804.00218/) (2018)
- [Graph Policy Gradients for Large Scale Robot Control](https://mlsanity.com/paper/1907.03822/) (2019)
- [Guided Policy Search as Approximate Mirror Descent](https://mlsanity.com/paper/1607.04614/) (2016)
- [Learning Neurosymbolic Generative Models via Program Synthesis](https://mlsanity.com/paper/1901.08565/) (2019)
- [Decentralization of Multiagent Policies by Learning What to Communicate](https://mlsanity.com/paper/1901.08490/) (2019)
- [Cooperative Team Strategies for Multi-player Perimeter-Defense Games](https://mlsanity.com/paper/1912.04342/) (2019)
- [Sample Complexity of Estimating the Policy Gradient for Nearly Deterministic Dynamical Systems](https://mlsanity.com/paper/1901.08562/) (2019)
- [Learning Safe Unlabeled Multi-Robot Planning with Motion Constraints](https://mlsanity.com/paper/1907.05300/) (2019)

## Built on by (most cited first, 13 of 13)

- [Compositional Reinforcement Learning from Logical Specifications](https://mlsanity.com/paper/2106.13906/) (2021)
- [Cooperative Exploration for Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2107.11444/) (2021)
- [Learning Transformer Programs](https://mlsanity.com/paper/2306.01128/) (2023)
- [Program Synthesis Guided Reinforcement Learning for Partially Observed Environments](https://mlsanity.com/paper/2102.11137/) (2021)
- [Uncertainty Quantification for Neurosymbolic Programs via Compositional Conformal Prediction](https://mlsanity.com/paper/2405.15912/) (2024)
- [Learning Connectivity for Data Distribution in Robot Teams](https://mlsanity.com/paper/2103.05091/) (2021)
- [Delayed Propagation Transformer: A Universal Computation Engine towards Practical Control in Cyber-Physical Systems](https://mlsanity.com/paper/2110.15926/) (2021)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Inductive Generalization in Reinforcement Learning from Specifications](https://mlsanity.com/paper/2406.03651/) (2024)
- [DR. WELL: Dynamic Reasoning and Learning with Symbolic World Model for Embodied LLM-Based Multi-Agent Collaboration](https://mlsanity.com/paper/2511.04646/) (2025)
- [Bridging Neural and Symbolic Representations with Transitional Dictionary Learning](https://mlsanity.com/paper/2308.02000/) (2023)
- [In Transformer We Trust? A Perspective on Transformer Architecture Failure Modes](https://mlsanity.com/paper/2602.14318/) (2026)
- [Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2603.15054/) (2026)

## Abstract (arXiv)

We study the problem of inferring communication structures that can solve cooperative multi-agent planning problems while minimizing the amount of communication. We quantify the amount of communication as the maximum degree of the communication graph; this metric captures settings where agents have limited bandwidth. Minimizing communication is challenging due to the combinatorial nature of both the decision space and the objective; for instance, we cannot solve this problem by training neural networks using gradient descent. We propose a novel algorithm that synthesizes a control policy that combines a programmatic communication policy used to generate the communication graph with a transformer policy network used to choose actions. Our algorithm first trains the transformer policy, which implicitly generates a "soft" communication graph; then, it synthesizes a programmatic communication policy that "hardens" this graph, forming a neurosymbolic transformer. Our experiments demonstrate how our approach can synthesize policies that generate low-degree communication graphs while maintaining near-optimal performance.