# ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind

arXiv: https://arxiv.org/abs/2111.09189 · HTML: https://mlsanity.com/paper/2111.09189/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 50 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.08839/) (2020)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Actor-Attention-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1810.02912/) (2018)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Graph Convolutional Reinforcement Learning](https://mlsanity.com/paper/1810.09202/) (2018)
- [Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration](https://mlsanity.com/paper/2010.09890/) (2020)
- [Opponent Modeling in Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05559/) (2016)
- [Causal Discovery in Physical Systems from Videos](https://mlsanity.com/paper/2007.00631/) (2020)
- [Modeling Others using Oneself in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.09640/) (2018)
- [Learning Individually Inferred Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/2006.06455/) (2020)
- [Learning Policy Representations in Multiagent Systems](https://mlsanity.com/paper/1806.06464/) (2018)
- [Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery](https://mlsanity.com/paper/1912.03558/) (2019)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)

## Built on by (most cited first, 30 of 58)

- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [ProAgent: Building Proactive Cooperative Agents with Large Language Models](https://mlsanity.com/paper/2308.11339/) (2023)
- [Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker](https://mlsanity.com/paper/2306.00924/) (2023)
- [Knowledge Mechanisms in Large Language Models: A Survey and Perspective](https://mlsanity.com/paper/2407.15017/) (2024)
- [Towards a Standardised Performance Evaluation Protocol for Cooperative MARL](https://mlsanity.com/paper/2209.10485/) (2022)
- [Learning Multi-Agent Communication from Graph Modeling Perspective](https://mlsanity.com/paper/2405.08550/) (2024)
- [Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy](https://mlsanity.com/paper/2407.06813/) (2024)
- [Language Models Represent Beliefs of Self and Others](https://mlsanity.com/paper/2402.18496/) (2024)
- [UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI](https://mlsanity.com/paper/2412.20977/) (2024)
- [Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2307.01158/) (2023)
- [RSPT: Reconstruct Surroundings and Predict Trajectories for Generalizable Active Object Tracking](https://mlsanity.com/paper/2304.03623/) (2023)
- [TarGF: Learning Target Gradient Field to Rearrange Objects without Explicit Goal Specification](https://mlsanity.com/paper/2209.00853/) (2022)
- [Proactive Multi-Camera Collaboration For 3D Human Pose Estimation](https://mlsanity.com/paper/2303.03767/) (2023)
- [MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems](https://mlsanity.com/paper/2505.18943/) (2025)
- [StackPlanner: A Centralized Hierarchical Multi-Agent System with Task-Experience Memory Management](https://mlsanity.com/paper/2601.05890/) (2026)
- [CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation](https://mlsanity.com/paper/2411.04679/) (2024)
- [Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.03022/) (2022)
- [Coordinate-Aligned Multi-Camera Collaboration for Active Multi-Object Tracking](https://mlsanity.com/paper/2202.10881/) (2022)
- [Fast Peer Adaptation with Context-aware Exploration](https://mlsanity.com/paper/2402.02468/) (2024)
- [Exponential Topology-enabled Scalable Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2502.19717/) (2025)
- [Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation](https://mlsanity.com/paper/2305.05116/) (2023)
- [Heterogeneous Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2307.13957/) (2023)
- [Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowds](https://mlsanity.com/paper/2405.16439/) (2024)
- [Explicit Modelling of Theory of Mind for Belief Prediction in Nonverbal Social Interactions](https://mlsanity.com/paper/2407.06762/) (2024)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2410.17373/) (2024)
- [Decentralized Collective World Model for Emergent Communication and Coordination](https://mlsanity.com/paper/2504.03353/) (2025)
- [TodyComm: Task-Oriented Dynamic Communication for Multi-Round LLM-based Multi-Agent System](https://mlsanity.com/paper/2602.03688/) (2026)

## Abstract (arXiv)

Being able to predict the mental states of others is a key factor to effective social interaction. It is also crucial for distributed multi-agent systems, where agents are required to communicate and cooperate. In this paper, we introduce such an important social-cognitive skill, i.e. Theory of Mind (ToM), to build socially intelligent agents who are able to communicate and cooperate effectively to accomplish challenging tasks. With ToM, each agent is capable of inferring the mental states and intentions of others according to its (local) observation. Based on the inferred states, the agents decide "when" and with "whom" to share their intentions. With the information observed, inferred, and received, the agents decide their sub-goals and reach a consensus among the team. In the end, the low-level executors independently take primitive actions to accomplish the sub-goals. We demonstrate the idea in two typical target-oriented multi-agent tasks: cooperative navigation and multi-sensor target coverage. The experiments show that the proposed model not only outperforms the state-of-the-art methods on reward and communication efficiency, but also shows good generalization across different scales of the environment.