mlsanityData to 14 Oct 2026

Can LLM-Augmented autonomous agents cooperate?, An evaluation of their cooperative capabilities through Melting Pot

arXiv 2403.11381 ↗First posted 18 Mar 2024cs.AI cs.CLcited by 11

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

As the field of AI continues to evolve, a significant dimension of this progression is the development of Large Language Models and their potential to enhance multi-agent artificial intelligence systems. This paper explores the cooperative capabilities of Large Language Model-augmented Autonomous Agents (LAAs) using the well-known Meltin Pot environments along with reference models such as GPT4 and GPT3.5. Preliminary results suggest that while these agents demonstrate a propensity for cooperation, they still struggle with effective collaboration in given environments, emphasizing the need for more robust architectures. The study's contributions include an abstraction layer to adapt Melting Pot game scenarios for LLMs, the implementation of a reusable architecture for LLM-mediated agent development - which includes short and long-term memories and different cognitive modules, and the evaluation of cooperation capabilities using a set of metrics tied to the Melting Pot's "Commons Harvest" game. The paper closes, by discussing the limitations of the current architectural framework and the potential of a new set of modules that fosters better cooperation among LAAs.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 21 of its 51 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

21 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges21 Jan
  2. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents11 Aug
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 17

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  7. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  8. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 222
  9. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  10. 2020Open Problems in Cooperative AIcited by 168
  11. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 110
  12. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 99
5 more
  1. 2023CompeteAI: Understanding the Competition Dynamics in Large Language Model-based Agentscited by 65
  2. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 63
  3. 2022Melting Pot 2.0cited by 40
  4. 2021A multi-agent reinforcement learning model of reputation and cooperation in human groupscited by 6
  5. 2023Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 4.

11 papers.

  1. 2024OASIS: Open Agent Social Interaction Simulations with One Million Agentscited by 132
  2. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  3. 2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 55
  4. 2026The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMscited by 10
  5. 2024IFAR: Multi-Perspective and Multi-Level Causal Discovery with LLMscited by 9
  6. 2024Cooperative Resilience in Artificial Intelligence Multiagent Systemscited by 8
  7. 2026Efficient and Interpretable Multi-Agent LLM Routing via Ant Colony Optimizationcited by 3
  8. 2025PRISON: Unmasking the Criminal Potential of Large Language Modelscited by 2
  9. 2025Is Your LLM Really Mastering the Concept? A Multi-Agent Benchmarkcited by 1
  10. 2026COOP2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systemscited by 0
  11. 2026Training Small LLMs as Spatial Multi-Agent Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  3. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures
  4. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents
  5. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  6. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  7. 2025LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey
  8. 2026More Capable, Less Cooperative? When LLMs Fail At Zero-Cost Collaboration