mlsanityData to 14 Oct 2026

Theory of Mind for Multi-Agent Collaboration via Large Language Models

arXiv 2310.10701 ↗First posted 16 Oct 2023cs.CL cs.AIcited by 135

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

While Large Language Models (LLMs) have demonstrated impressive accomplishments in both reasoning and planning, their abilities in multi-agent collaborations remains largely unexplored. This study evaluates LLM-based agents in a multi-agent cooperative text game with Theory of Mind (ToM) inference tasks, comparing their performance with Multi-Agent Reinforcement Learning (MARL) and planning-based baselines. We observed evidence of emergent collaborative behaviors and high-order Theory of Mind capabilities among LLM-based agents. Our results reveal limitations in LLM-based agents' planning optimization due to systematic failures in managing long-horizon contexts and hallucination about the task state. We explore the use of explicit belief state representations to mitigate these issues, finding that it enhances task performance and the accuracy of ToM inferences for LLM-based agents.

Key facts. Cited by 135 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 23 of its 47 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

23 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents3 Feb

Other references · 22

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  7. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  8. 2016OpenAI Gymcited by 1,736
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  11. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  12. 2019The StarCraft Multi-Agent Challengecited by 597
10 more
  1. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  2. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  3. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  4. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  5. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 133
  6. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  7. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  8. 2023Theory of Mind as Intrinsic Motivation for Multi-Agent Reinforcement Learningcited by 19
  9. 2021Emergence of Theory of Mind Collaboration in Multiagent Systemscited by 8
  10. 2020Improving Multi-Agent Cooperation using Theory of Mindcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 35; 2025: 56; 2026: 42.

135 papers.

Sort:

By area:

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  2. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 316
  3. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  4. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  5. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  6. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  7. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 83
  8. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 82
  9. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  10. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
  11. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  12. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 49
38 more
  1. 2024A Scalable Communication Protocol for Networks of Large Language Modelscited by 45
  2. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  3. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  4. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  5. 2025PiFlow: Principle-Aware Scientific Discovery with Multi-Agent Collaborationcited by 28
  6. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  7. 2025Large Language Models Miss the Multi-Agent Markcited by 26
  8. 2024Multi-Agent Large Language Models for Conversational Task-Solvingcited by 24
  9. 2024ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaborationcited by 24
  10. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  11. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yetcited by 23
  12. 2024Why Solving Multi-agent Path Finding with Large Language Model has not Succeeded Yetcited by 22
  13. 2026Large Language Model Reasoning Failurescited by 21
  14. 2024Evaluating and Enhancing LLMs Agent based on Theory of Mind in Guandan: A Multi-Player Cooperative Game under Imperfect Informationcited by 20
  15. 2024Towards Effective GenAI Multi-Agent Collaboration: Design and Evaluation for Enterprise Applicationscited by 19
  16. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  17. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  18. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  19. 2025SoMi-ToM: Evaluating Multi-Perspective Theory of Mind in Embodied Social Interactionscited by 16
  20. 2025Self-Resource Allocation in Multi-Agent LLM Systemscited by 15
  21. 2024Do LLMs have Consistent Values?cited by 14
  22. 2024Evolving AI Collectives to Enhance Human Diversity and Enable Self-Regulationcited by 13
  23. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflectioncited by 13
  24. 2025TOM-SWE: User Mental Modeling For Software Engineering Agentscited by 13
  25. 2026Prime Agent: A Self-Improving RLM Harnesscited by 13
  26. 2024CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperationcited by 12
  27. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  28. 2025Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agentscited by 12
  29. 2025Learning from Diverse Reasoning Paths with Routing and Collaborationcited by 12
  30. 2024Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialoguescited by 11
  31. 2024Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Datacited by 10
  32. 2025Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Surveycited by 10
  33. 2025Benchmarking LLMs' Swarm intelligencecited by 10
  34. 2025United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theorycited by 10
  35. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learningcited by 9
  36. 2025V-Stylist: Video Stylization via Collaboration and Reflection of MLLM Agentscited by 9
  37. 2025Bayesian Social Deduction with Graph-Informed Language Modelscited by 9
  38. 2025GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysiscited by 8

50 most cited of 135 All 135 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  2. 2026Adaptive Theory of Mind for LLM-based Multi-Agent Coordination
  3. 2023LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models
  4. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
  5. 2025LLM-Hanabi: Evaluating Multi-Agent Gameplays with Theory-of-Mind and Rationale Inference in Imperfect Information Collaboration Game
  6. 2024Mental Modeling of Reinforcement Learning Agents by Language Models
  7. 2025Collaborative Belief Reasoning with LLMs for Efficient Multi-Agent Collaboration
  8. 2026MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs