mlsanityData to 7 Oct 2026

Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models

arXiv 2407.07086 ↗First posted 9 Jul 2024cs.AIcited by 45

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Multi-agent reinforcement learning (MARL) methods struggle with the non-stationarity of multi-agent systems and fail to adaptively learn online when tested with novel agents. Here, we leverage large language models (LLMs) to create an autonomous agent that can handle these challenges. Our agent, Hypothetical Minds, consists of a cognitively-inspired architecture, featuring modular components for perception, memory, and hierarchical planning over two levels of abstraction. We introduce the Theory of Mind module that scaffolds the high-level planning process by generating hypotheses about other agents' strategies in natural language. It then evaluates and iteratively refines these hypotheses by reinforcing hypotheses that make correct predictions about the other agents' behavior. Hypothetical Minds significantly improves performance over previous LLM-agent and RL baselines on a range of competitive, mixed motive, and collaborative domains in the Melting Pot benchmark, including both dyadic and population-based environments. Additionally, comparisons against LLM-agent baselines and ablations reveal the importance of hypothesis evaluation and refinement for succeeding on complex scenarios.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 59 references that are in the map. First posted on arXiv 9 Jul 2024.

Builds on

31 of 59 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents20 Nov
  2. 2023Cognitive Architectures for Language Agents5 Sep
  3. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  4. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Models22 Aug
  5. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents11 Aug
  6. 2023Building Cooperative Embodied Agents Modularly with Large Language Models5 Jul

Other references · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  11. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  12. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
13 more
  1. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  2. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 217
  3. 2023Playing repeated games with Large Language Modelscited by 213
  4. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  5. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
  6. 2023Understanding Social Reasoning in Language Models with Language Modelscited by 166
  7. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
  8. 2023Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinementcited by 89
  9. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  10. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  11. 2023Turning large language models into cognitive modelscited by 66
  12. 2023Learning to Model the World with Languagecited by 66
  13. 2022Melting Pot 2.0cited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 22; 2026: 22.

45 papers.

Sort:

By area:

  1. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  2. 2024A Scalable Communication Protocol for Networks of Large Language Modelscited by 43
  3. 2025Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Modelscited by 28
  4. 2025SPIN-Bench: How Well Do LLMs Plan Strategically and Reason Socially?cited by 24
  5. 2025Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systemscited by 22
  6. 2026Large Language Model Reasoning Failurescited by 20
  7. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  8. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  9. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 16
  10. 2025Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Modelscited by 14
  11. 2025Self-Resource Allocation in Multi-Agent LLM Systemscited by 13
  12. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systemscited by 12
33 more
  1. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  2. 2025WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-makingcited by 10
  3. 2025Do Large Language Models Have a Planning Theory of Mind? Evidence from MindGames: a Multi-Step Persuasion Taskcited by 9
  4. 2025Persuasion Should be Double-Blind: A Multi-Domain Dialogue Dataset With Faithfulness Based on Causal Theory of Mindcited by 8
  5. 2025Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Flycited by 6
  6. 2025Reasoning and Behavioral Equilibria in LLM-Nash Games: From Mindsets to Actionscited by 6
  7. 2025Towards Machine Theory of Mind with Large Language Model-Augmented Inverse Planningcited by 4
  8. 2026MetaMind: General and Cognitive World Models in Multi-Agent Systems by Meta-Theory of Mindcited by 3
  9. 2025CausalPlan: Empowering Efficient LLM Multi-Agent Collaboration Through Causality-Driven Planningcited by 2
  10. 2026Discovering Differences in Strategic Behavior Between Humans and LLMscited by 2
  11. 2026CONSCIENTIA: Can LLM Agents Learn to Strategize? Emergent Deception and Trust in a Multi-Agent NYC Simulationcited by 2
  12. 2026Think Thrice Before You Speak: Dual knowledge-enhanced Theory-of-Mind Reasoning for Persuasive Agentscited by 2
  13. 2026When Should Models Change Their Minds? Contextual Belief Management in Large Language Modelscited by 2
  14. 2025The challenge of hidden gifts in multi-agent reinforcement learningcited by 1
  15. 2025ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedbackcited by 1
  16. 2026GameTalk: Training LLMs for Strategic Conversationcited by 1
  17. 2026SIDiffAgent: Self-Improving Diffusion Agentcited by 1
  18. 2026DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contentioncited by 1
  19. 2026A formal definition and meta-model for a machine theory of mindcited by 1
  20. 2025Validating Generative Agent-Based Models of Social Norm Enforcement: From Replication to Novel Predictionscited by 0
  21. 2025Understanding Human Limits in Pattern Recognition: A Computational Model of Sequential Reasoning in Rock, Paper, Scissorscited by 0
  22. 2025ToMPO: Training LLM Strategic Decision Making from a Multi-Agent Perspectivecited by 0
  23. 2026A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Datacited by 0
  24. 2026On the Strengths and Weaknesses of Data for Open-set Embodied Assistancecited by 0
  25. 2026EpiPersona: Persona Projection and Episode Coupling for Pluralistic Preference Modelingcited by 0
  26. 2026Shadow-Loom: Causal Reasoning over Graphical World Models of Narrativescited by 0
  27. 2026SODE: Analyzing Social Dynamics in LLM Agentscited by 0
  28. 2026User-Aware Active Knowledge Acquisition for Emotional Support Dialoguecited by 0
  29. 2026MindZero: Learning Online Mental Reasoning With Zero Annotationscited by 0
  30. 2026Enhancing Decision-Making with Large Language Models through Multi-Agent Fictitious Playcited by 0
  31. 2026RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experimentscited by 0
  32. 2026Theory of Scene: Breaking the Symmetry Trap in Multi-Agent LLM Coordinationcited by 0
  33. 2026SAGE: Structured Strategic Reasoning for Efficient LLM Game Playingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Theory of Mind for Multi-Agent Collaboration via Large Language Models
  2. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  3. 2024Mental Modeling of Reinforcement Learning Agents by Language Models
  4. 2025Unified Mind Model: Reimagining Autonomous Agents in the LLM Era
  5. 2025Towards Cognitive Synergy in LLM-Based Multi-Agent Systems: Integrating Theory of Mind and Critical Evaluation
  6. 2026Rethinking Agentic Reinforcement Learning In Large Language Models
  7. 2026Adaptive Theory of Mind for LLM-based Multi-Agent Coordination
  8. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Information