# Few-shot Language Coordination by Modeling Theory of Mind

arXiv: https://arxiv.org/abs/2107.05697 · HTML: https://mlsanity.com/paper/2107.05697/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 52 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models](https://mlsanity.com/paper/1805.12114/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- ["Other-Play" for Zero-Shot Coordination](https://mlsanity.com/paper/2003.02979/) (2020)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Ease-of-Teaching and Language Structure from Emergent Communication](https://mlsanity.com/paper/1906.02403/) (2019)
- [Evaluating Theory of Mind in Question Answering](https://mlsanity.com/paper/1808.09352/) (2018)
- [Emergent Communication through Negotiation](https://mlsanity.com/paper/1804.03980/) (2018)
- [Unified Pragmatic Models for Generating and Following Instructions](https://mlsanity.com/paper/1711.04987/) (2017)
- [Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networks](https://mlsanity.com/paper/1605.07127/) (2016)
- [How agents see things: On visual representations in an emergent language game](https://mlsanity.com/paper/1808.10696/) (2018)
- [Learning Language Games through Interaction](https://mlsanity.com/paper/1606.02447/) (2016)
- [A mathematical theory of cooperative communication](https://mlsanity.com/paper/1910.02822/) (2019)
- [Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations](https://mlsanity.com/paper/2010.15896/) (2020)
- [Neural Recursive Belief States in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2102.02274/) (2021)
- [Emergence of Pragmatics from Referential Game between Theory of Mind Agents](https://mlsanity.com/paper/2001.07752/) (2020)
- [Learning in the Rational Speech Acts Model](https://mlsanity.com/paper/1510.06807/) (2015)
- [Shaping representations through communication: community size effect in artificial learning systems](https://mlsanity.com/paper/1912.06208/) (2019)

## Built on by (most cited first, 30 of 32)

- [SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://mlsanity.com/paper/2310.11667/) (2023)
- [Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMs](https://mlsanity.com/paper/2210.13312/) (2022)
- [Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Tracker](https://mlsanity.com/paper/2306.00924/) (2023)
- [SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents](https://mlsanity.com/paper/2403.08715/) (2024)
- [LLMs achieve adult human performance on higher-order theory of mind tasks](https://mlsanity.com/paper/2405.18870/) (2024)
- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [CooperBench: Why Coding Agents Cannot be Your Teammates Yet](https://mlsanity.com/paper/2601.13295/) (2026)
- [I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragons](https://mlsanity.com/paper/2212.10060/) (2022)
- [Meta-Learning for Multi-Label Few-Shot Classification](https://mlsanity.com/paper/2110.13494/) (2021)
- [Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mind](https://mlsanity.com/paper/2211.04684/) (2022)
- [Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMs](https://mlsanity.com/paper/2408.01417/) (2024)
- [Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalization](https://mlsanity.com/paper/2306.09299/) (2023)
- [Computational Language Acquisition with Theory of Mind](https://mlsanity.com/paper/2303.01502/) (2023)
- [Learning to Mediate Disparities Towards Pragmatic Communication](https://mlsanity.com/paper/2203.13685/) (2022)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [A Framework for Learning to Request Rich and Contextually Useful Information from Humans](https://mlsanity.com/paper/2110.08258/) (2021)
- [Learning Robust Real-Time Cultural Transmission without Human Data](https://mlsanity.com/paper/2203.00715/) (2022)
- [Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue](https://mlsanity.com/paper/2305.11271/) (2023)
- [Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation](https://mlsanity.com/paper/2504.16060/) (2025)
- [Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Models](https://mlsanity.com/paper/2301.05149/) (2022)
- [Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind](https://mlsanity.com/paper/2305.19933/) (2023)
- [Fast Peer Adaptation with Context-aware Exploration](https://mlsanity.com/paper/2402.02468/) (2024)
- [Success and Cost Elicit Convention Formation for Efficient Communication](https://mlsanity.com/paper/2510.24023/) (2025)
- [Adaptive Theory of Mind for LLM-based Multi-Agent Coordination](https://mlsanity.com/paper/2603.16264/) (2026)
- [XToM: Exploring the Multilingual Theory of Mind for Large Language Models](https://mlsanity.com/paper/2506.02461/) (2025)
- [BEDA: Belief Estimation as Probabilistic Constraints for Performing Strategic Dialogue Acts](https://mlsanity.com/paper/2512.24885/) (2025)
- [Core Challenges in Embodied Vision-Language Planning](https://mlsanity.com/paper/2304.02738/) (2023)
- [Do LLM Agents Mirror Socio-Cognitive Effects in Power-Asymmetric Conversations?](https://mlsanity.com/paper/2605.17694/) (2026)
- [A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Data](https://mlsanity.com/paper/2608.08663/) (2026)
- [A Primer on Computational Semantics for Artificial Intelligence Systems](https://mlsanity.com/paper/2608.25022/) (2026)

## Abstract (arXiv)

textitNo man is an island. Humans communicate with a large community by coordinating with different interlocutors within short conversations. This ability has been understudied by the research on building neural communicative agents. We study the task of few-shot textitlanguage coordination: agents quickly adapting to their conversational partners' language abilities. Different from current communicative agents trained with self-play, we require the lead agent to coordinate with a textitpopulation of agents with different linguistic abilities, quickly adapting to communicate with unseen agents in the population. This requires the ability to model the partner's beliefs, a vital component of human communication. Drawing inspiration from theory-of-mind (ToM; Premack& Woodruff (1978)), we study the effect of the speaker explicitly modeling the listeners' mental states. The speakers, as shown in our experiments, acquire the ability to predict the reactions of their partner, which helps it generate instructions that concisely express its communicative goal. We examine our hypothesis that the instructions generated with ToM modeling yield better communication performance in both a referential game and a language navigation task. Positive results from our experiments hint at the importance of explicitly modeling communication as a socio-pragmatic progress.