# An Efficient Open World Environment for Multi-Agent Social Learning

arXiv: https://arxiv.org/abs/2508.15679 · HTML: https://mlsanity.com/paper/2508.15679/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 17 of 17)

- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [MineRL: A Large-Scale Dataset of Minecraft Demonstrations](https://mlsanity.com/paper/1907.13440/) (2019)
- [Benchmarking the Spectrum of Agent Capabilities](https://mlsanity.com/paper/2109.06780/) (2021)
- [Discovered Policy Optimisation](https://mlsanity.com/paper/2210.05639/) (2022)
- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning](https://mlsanity.com/paper/2402.16801/) (2024)
- [Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents](https://mlsanity.com/paper/1903.00784/) (2019)
- [Emergent Social Learning via Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2010.00581/) (2020)
- [VillagerAgent: A Graph-Based Multi-Agent Framework for Coordinating Complex Task Dependencies in Minecraft](https://mlsanity.com/paper/2406.05720/) (2024)
- [PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning](https://mlsanity.com/paper/2102.12560/) (2021)
- [SEIHAI: A Sample-efficient Hierarchical AI for the MineRL Competition](https://mlsanity.com/paper/2111.08857/) (2021)
- [The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design](https://mlsanity.com/paper/2406.17949/) (2024)
- [InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma](https://mlsanity.com/paper/2411.09856/) (2024)

## Built on by (most cited first, 3 of 3)

- [Benchmarking Open-Ended Multi-Agent Coordination in Language Agents](https://mlsanity.com/paper/2606.08340/) (2026)
- [Toward Virtuous Reinforcement Learning: A Critique and Roadmap](https://mlsanity.com/paper/2512.04246/) (2025)
- [Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets](https://mlsanity.com/paper/2602.16062/) (2026)

## Abstract (arXiv)

Many challenges remain before AI agents can be deployed in real-world environments. However, one virtue of such environments is that they are inherently multi-agent and contain human experts. Using advanced social intelligence in such an environment can help an AI agent learn adaptive skills and behaviors that a known expert exhibits. While social intelligence could accelerate training, it is currently difficult to study due to the lack of open-ended multi-agent environments. In this work, we present an environment in which multiple self-interested agents can pursue complex and independent goals, reflective of real world challenges. This environment will enable research into the development of socially intelligent AI agents in open-ended multi-agent settings, where agents may be implicitly incentivized to cooperate to defeat common enemies, build and share tools, and achieve long horizon goals. In this work, we investigate the impact on agent performance due to social learning in the presence of experts and implicit cooperation such as emergent collaborative tool use, and whether agents can benefit from either cooperation or competition in this environment.