mlsanityData to 7 Oct 2026

Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learning

arXiv 2307.01158 ↗First posted 3 Jul 2023cs.LG cs.AI cs.MAcited by 19

Not on the map. Why?

Abstract (arXiv)

The ability to model the mental states of others is crucial to human social intelligence, and can offer similar benefits to artificial agents with respect to the social dynamics induced in multi-agent settings. We present a method of grounding semantically meaningful, human-interpretable beliefs within policies modeled by deep networks. We then consider the task of 2nd-order belief prediction. We propose that ability of each agent to predict the beliefs of the other agents can be used as an intrinsic reward signal for multi-agent reinforcement learning. Finally, we present preliminary empirical results in a mixed cooperative-competitive environment.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

24 of 36 references.

References · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  6. 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
  7. 2022Post-hoc Concept Bottleneck Modelscited by 289
  8. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  9. 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 233
  10. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  11. 2018Machine Theory of Mindcited by 222
  12. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
12 more
  1. 2016Opponent Modeling in Deep Reinforcement Learningcited by 94
  2. 2019Influence-Based Multi-Agent Explorationcited by 75
  3. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  4. 2021ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mindcited by 57
  5. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learningcited by 36
  6. 2022Mind the gap: Challenges of deep learning approaches to Theory of Mindcited by 21
  7. 2023Concept Learning for Interpretable Multi-Agent Reinforcement Learningcited by 18
  8. 2021Deep Interpretable Models of Theory of Mindcited by 13
  9. 2018Intent-aware Multi-agent Reinforcement Learningcited by 12
  10. 2021Theory of Mind for Deep Reinforcement Learning in Hanabicited by 11
  11. 2023Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoningcited by 11
  12. 2021Learning Human Rewards by Inferring Their Latent Intelligence Levels in Multi-Agent Games: A Theory-of-Mind Approach with Application to Driving Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 10; 2026: 4.

19 papers.

Sort:

By area:

  1. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Modelscited by 129
  2. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Informationcited by 20
  3. 2024Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisitioncited by 10
  4. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflectioncited by 10
  5. 2026Factored Latent Action World Modelscited by 6
  6. 2023Benchmarking and Enhancing Disentanglement in Concept-Residual Modelscited by 5
  7. 2025From Black Boxes to Transparent Minds: Evaluating and Enhancing the Theory of Mind in Multimodal Large Language Modelscited by 5
  8. 2025Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planningcited by 4
  9. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  10. 2025Can LLM Agents Solve Collaborative Tasks? A Study on Urgency-Aware Planning and Coordinationcited by 3
  11. 2026MetaMind: General and Cognitive World Models in Multi-Agent Systems by Meta-Theory of Mindcited by 3
  12. 2025BeliefNest: A Joint Action Simulator for Embodied Agents with Theory of Mindcited by 1
7 more
  1. 2025Modeling Latent Partner Strategies for Adaptive Zero-Shot Human-Agent Collaborationcited by 1
  2. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 1
  3. 2025Doing Things with Words: Rethinking Theory of Mind Simulation in Large Language Modelscited by 1
  4. 2026A formal definition and meta-model for a machine theory of mindcited by 1
  5. 2026Robots Influencing Humans to Reveal their Goals during Collaboration and Competitioncited by 1
  6. 2024Autonomous Vehicle Decision-Making Framework for Considering Malicious Behavior at Unsignalized Intersectionscited by 0
  7. 2025Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaborationcited by 0