mlsanityData to 7 Oct 2026

Few-shot Language Coordination by Modeling Theory of Mind

arXiv 2107.05697 ↗First posted 12 Jul 2021cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

textitNo man is an island. Humans communicate with a large community by coordinating with different interlocutors within short conversations. This ability has been understudied by the research on building neural communicative agents. We study the task of few-shot textitlanguage coordination: agents quickly adapting to their conversational partners' language abilities. Different from current communicative agents trained with self-play, we require the lead agent to coordinate with a textitpopulation of agents with different linguistic abilities, quickly adapting to communicate with unseen agents in the population. This requires the ability to model the partner's beliefs, a vital component of human communication. Drawing inspiration from theory-of-mind (ToM; Premack& Woodruff (1978)), we study the effect of the speaker explicitly modeling the listeners' mental states. The speakers, as shown in our experiments, acquire the ability to predict the reactions of their partner, which helps it generate instructions that concisely express its communicative goal. We examine our hypothesis that the instructions generated with ToM modeling yield better communication performance in both a referential game and a language navigation task. Positive results from our experiments hint at the importance of explicitly modeling communication as a socio-pragmatic progress.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

26 of 52 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  4. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  8. 2018How to train your MAMLcited by 306
  9. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  10. 2018Machine Theory of Mindcited by 222
  11. 2020"Other-Play" for Zero-Shot Coordinationcited by 194
  12. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
14 more
  1. 2020Compositionality and Generalization in Emergent Languagescited by 103
  2. 2019Ease-of-Teaching and Language Structure from Emergent Communicationcited by 65
  3. 2018Evaluating Theory of Mind in Question Answeringcited by 64
  4. 2018Emergent Communication through Negotiationcited by 62
  5. 2017Unified Pragmatic Models for Generating and Following Instructionscited by 55
  6. 2016Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networkscited by 49
  7. 2018How agents see things: On visual representations in an emergent language gamecited by 45
  8. 2016Learning Language Games through Interactioncited by 43
  9. 2019A mathematical theory of cooperative communicationcited by 23
  10. 2020Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populationscited by 18
  11. 2021Neural Recursive Belief States in Multi-Agent Reinforcement Learningcited by 18
  12. 2020Emergence of Pragmatics from Referential Game between Theory of Mind Agentscited by 17
  13. 2015Learning in the Rational Speech Acts Modelcited by 16
  14. 2019Shaping representations through communication: community size effect in artificial learning systemscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 9; 2024: 4; 2025: 4; 2026: 7.

32 papers.

Sort:

By area:

  1. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agentscited by 350
  2. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  3. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  4. 2024SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agentscited by 89
  5. 2024LLMs achieve adult human performance on higher-order theory of mind taskscited by 59
  6. 2023Towards A Holistic Landscape of Situated Theory of Mind in Large Language Modelscited by 32
  7. 2022I Cast Detect Thoughts: Learning to Converse and Guide with Intents and Theory-of-Mind in Dungeons and Dragonscited by 22
  8. 2021Meta-Learning for Multi-Label Few-Shot Classificationcited by 21
  9. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yetcited by 20
  10. 2022Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mindcited by 19
  11. 2024Talk Less, Interact Better: Evaluating In-context Conversational Adaptation in Multimodal LLMscited by 18
  12. 2023Computational Language Acquisition with Theory of Mindcited by 15
20 more
  1. 2023Can Language Models Teach Weaker Agents? Teacher Explanations Improve Students via Personalizationcited by 15
  2. 2022Learning to Mediate Disparities Towards Pragmatic Communicationcited by 14
  3. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  4. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  5. 2022Learning Robust Real-Time Cultural Transmission without Human Datacited by 9
  6. 2023Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialoguecited by 9
  7. 2025Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generationcited by 8
  8. 2022Define, Evaluate, and Improve Task-Oriented Cognitive Capabilities for Instruction Generation Modelscited by 7
  9. 2023Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mindcited by 7
  10. 2024Fast Peer Adaptation with Context-aware Explorationcited by 7
  11. 2025Success and Cost Elicit Convention Formation for Efficient Communicationcited by 7
  12. 2026Adaptive Theory of Mind for LLM-based Multi-Agent Coordinationcited by 3
  13. 2025XToM: Exploring the Multilingual Theory of Mind for Large Language Modelscited by 2
  14. 2025BEDA: Belief Estimation as Probabilistic Constraints for Performing Strategic Dialogue Actscited by 2
  15. 2023Core Challenges in Embodied Vision-Language Planningcited by 0
  16. 2026Do LLM Agents Mirror Socio-Cognitive Effects in Power-Asymmetric Conversations?cited by 0
  17. 2026A Dynamic-Semantics Framework for Grounding Human Referring Expressions in Visual Perceptual Datacited by 0
  18. 2026A Primer on Computational Semantics for Artificial Intelligence Systemscited by 0
  19. 2026Prospective Interpretation Risk: Principled Communication Control Between LLMscited by 0
  20. 2026Reading the Room: Foundations, Design, and Challenges of Normative Competence in LLMscited by 0