# Emergence of Theory of Mind Collaboration in Multiagent Systems

arXiv: https://arxiv.org/abs/2110.00121 · HTML: https://mlsanity.com/paper/2110.00121/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 25 of 25)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/1709.04326/) (2017)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017)
- [Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1702.08887/) (2017)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017)
- [Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1811.01458/) (2018)
- [Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1901.09207/) (2019)
- [Pragmatic-Pedagogic Value Alignment](https://mlsanity.com/paper/1707.06354/) (2017)
- [Compositional Obverter Communication Learning From Raw Visual Input](https://mlsanity.com/paper/1804.02341/) (2018)
- [Multiagent Soft Q-Learning](https://mlsanity.com/paper/1804.09817/) (2018)
- [Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networks](https://mlsanity.com/paper/1602.02672/) (2016)
- [Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning](https://mlsanity.com/paper/1611.01722/) (2016)
- [Monte Carlo Sampling Methods for Approximating Interactive POMDPs](https://mlsanity.com/paper/1401.3455/) (2014)
- [Emergent Communication in a Multi-Modal, Multi-Step Referential Game](https://mlsanity.com/paper/1705.10369/) (2017)

## Built on by (most cited first, 8 of 8)

- [Theory of Mind for Multi-Agent Collaboration via Large Language Models](https://mlsanity.com/paper/2310.10701/) (2023)
- [Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information](https://mlsanity.com/paper/2408.02559/) (2024)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks](https://mlsanity.com/paper/2309.13223/) (2023)
- [Reasoning with the Theory of Mind for Pragmatic Semantic Communication](https://mlsanity.com/paper/2311.18224/) (2023)
- [Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordination](https://mlsanity.com/paper/2508.14635/) (2025)
- [Using High-Level Patterns to Estimate How Humans Predict a Robot will Behave](https://mlsanity.com/paper/2409.13533/) (2024)
- [A Primer on Computational Semantics for Artificial Intelligence Systems](https://mlsanity.com/paper/2608.25022/) (2026)

## Abstract (arXiv)

Currently, in the study of multiagent systems, the intentions of agents are usually ignored. Nonetheless, as pointed out by Theory of Mind (ToM), people regularly reason about other's mental states, including beliefs, goals, and intentions, to obtain performance advantage in competition, cooperation or coalition. However, due to its intrinsic recursion and intractable modeling of distribution over belief, integrating ToM in multiagent planning and decision making is still a challenge. In this paper, we incorporate ToM in multiagent partially observable Markov decision process (POMDP) and propose an adaptive training algorithm to develop effective collaboration between agents with ToM. We evaluate our algorithms with two games, where our algorithm surpasses all previous decentralized execution algorithms without modeling ToM.