# AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making

arXiv: https://arxiv.org/abs/2411.03865 · HTML: https://mlsanity.com/paper/2411.03865/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use.
- Builds on 38 of its 89 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Placed in

- LLM agents and tool use / [Reinforcement learning for LLM agents](https://mlsanity.com/agents/reinforcement-learning-for-llm-agents/)

## Builds on (most cited first, 30 of 38)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Quantifying Generalization in Reinforcement Learning](https://mlsanity.com/paper/1812.02341/) (2018)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory](https://mlsanity.com/paper/2305.17144/) (2023)
- [Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design](https://mlsanity.com/paper/2012.02096/) (2020)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Benchmarking the Spectrum of Agent Capabilities](https://mlsanity.com/paper/2109.06780/) (2021)
- [Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents](https://mlsanity.com/paper/1802.08757/) (2018)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Open-Ended Learning Leads to Generally Capable Agents](https://mlsanity.com/paper/2107.12808/) (2021)
- [JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models](https://mlsanity.com/paper/2311.05997/) (2023)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems](https://mlsanity.com/paper/1709.08071/) (2017)
- [Deep Coordination Graphs](https://mlsanity.com/paper/1910.00091/) (2019)
- [MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence](https://mlsanity.com/paper/1712.00600/) (2017)
- [Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2006.11438/) (2020)
- [Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward](https://mlsanity.com/paper/2006.06626/) (2020)
- [Melting Pot 2.0](https://mlsanity.com/paper/2211.13746/) (2022)
- [No-Regret Learning in Time-Varying Zero-Sum Games](https://mlsanity.com/paper/2201.12736/) (2022)
- [OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents](https://mlsanity.com/paper/2407.00114/) (2024)
- [Sample-Efficient Reinforcement Learning of Partially Observable Markov Games](https://mlsanity.com/paper/2206.01315/) (2022)
- [Learning Structured Communication for Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2002.04235/) (2020)
- [Stabilizing Unsupervised Environment Design with a Learned Adversary](https://mlsanity.com/paper/2308.10797/) (2023)

## Built on by (most cited first, 20 of 20)

- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Application](https://mlsanity.com/paper/2412.17481/) (2024)
- [Heterogeneous Swarms: Jointly Optimizing Model Roles and Weights for Multi-LLM Systems](https://mlsanity.com/paper/2502.04510/) (2025)
- [SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning](https://mlsanity.com/paper/2506.05425/) (2025)
- [GNNs as Predictors of Agentic Workflow Performances](https://mlsanity.com/paper/2503.11301/) (2025)
- [Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective](https://mlsanity.com/paper/2501.00581/) (2024)
- [Topological Structure Learning Should Be A Research Priority for LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2505.22467/) (2025)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)
- [Games Agents Play: Towards Transactional Analysis in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2507.21354/) (2025)
- [Social World Model-Augmented Mechanism Design Policy Learning](https://mlsanity.com/paper/2510.19270/) (2025)
- [Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Tasks](https://mlsanity.com/paper/2509.05651/) (2025)
- [SimCity: Multi-Agent Urban Development Simulation with Rich Interactions](https://mlsanity.com/paper/2510.01297/) (2025)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [TACLA: An LLM-Based Multi-Agent Tool for Transactional Analysis Training in Education](https://mlsanity.com/paper/2510.17913/) (2025)
- [IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment](https://mlsanity.com/paper/2506.12331/) (2025)
- [On the Role of Contextual Information and Ego States in LLM Agent Behavior for Transactional Analysis Dialogues](https://mlsanity.com/paper/2512.17060/) (2025)
- [Adaptive Punishment for Cooperation in Mixed-Motive Games](https://mlsanity.com/paper/2605.24516/) (2026)
- [SidConArena: An Environment Evaluating Agents in Open-Ended,Positive-Sum Bargaining Game](https://mlsanity.com/paper/2606.27397/) (2026)
- [From Optimal Policies to Individual Differences: Rethinking Reinforcement Learning for Biology](https://mlsanity.com/paper/2607.16542/) (2026)
- [Grounded Normative Rule Generation with Structured Search](https://mlsanity.com/paper/2608.22229/) (2026)

## Abstract (arXiv)

Traditional interactive environments limit agents' intelligence growth with fixed tasks. Recently, single-agent environments address this by generating new tasks based on agent actions, enhancing task diversity. We consider the decision-making problem in multi-agent settings, where tasks are further influenced by social connections, affecting rewards and information access. However, existing multi-agent environments lack a combination of adaptive physical surroundings and social connections, hindering the learning of intelligent behaviors. To address this, we introduce AdaSociety, a customizable multi-agent environment featuring expanding state and action spaces, alongside explicit and alterable social structures. As agents progress, the environment adaptively generates new tasks with social structures for agents to undertake. In AdaSociety, we develop three mini-games showcasing distinct social structures and tasks. Initial results demonstrate that specific social structures can promote both individual and collective benefits, though current reinforcement learning and LLM-based algorithms show limited effectiveness in leveraging social structures to enhance performance. Overall, AdaSociety serves as a valuable research platform for exploring intelligence in diverse physical and social settings. The code is available at https://github.com/bigai-ai/AdaSociety.