mlsanityData to 7 Oct 2026

Static Sandboxes Are Inadequate: Modeling Societal Complexity Requires Open-Ended Co-Evolution in LLM-Based Multi-Agent Simulations

arXiv 2510.13982 ↗First posted 15 Oct 2025cs.MA cs.AIcited by 2

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

What if artificial agents could not just communicate, but also evolve, adapt, and reshape their worlds in ways we cannot fully predict? With llm now powering multi-agent systems and social simulations, we are witnessing new possibilities for modeling open-ended, ever-changing environments. Yet, most current simulations remain constrained within static sandboxes, characterized by predefined tasks, limited dynamics, and rigid evaluation criteria. These limitations prevent them from capturing the complexity of real-world societies. In this paper, we argue that static, task-specific benchmarks are fundamentally inadequate and must be rethought. We critically review emerging architectures that blend llm with multi-agent dynamics, highlight key hurdles such as balancing stability and diversity, evaluating unexpected behaviors, and scaling to greater complexity, and introduce a fresh taxonomy for this rapidly evolving field. Finally, we present a research roadmap centered on open-endedness, continuous co-evolution, and the development of resilient, socially aligned AI ecosystems. We call on the community to move beyond static paradigms and help shape the next generation of adaptive, socially-aware multi-agent simulations.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 87 of its 99 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

87 of 99 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulations4 Apr
  2. 2025Are Human Interactions Replicable by Generative Agents? A Case Study on Pronoun Usage in Hierarchical Interactions25 Jan
  3. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents4 Dec
  4. 2024Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model21 Nov
  5. 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals15 Nov
  6. 2024Mimicking the Mavens: Agent-based Opinion Synthesis and Emotion Prediction for Social Media Influencers30 Jul
  7. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents25 Apr
  8. 2024Large language model empowered participatory urban planning24 Jan
  9. 2023Generative agents in the streets: Exploring the use of Large Language Models (LLMs) in collecting urban perceptions20 Dec
  10. 2023Simulating Public Administration Crisis: A Novel Generative Agent-Based Simulation System to Lower Technology Barriers in Social Science Research12 Nov
  11. 2023Character-LLM: A Trainable Agent for Role-Playing16 Oct
  12. 2023The Role of Summarization in Generative Agents: A Preliminary Perspective2 May
1 more
  1. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 74

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  7. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 752
  8. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  9. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
  10. 2023ToolQA: A Dataset for LLM Question Answering with External Toolscited by 273
  11. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  12. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
38 more
  1. 2025Survey on Evaluation of LLM-based Agentscited by 190
  2. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  3. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  4. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  5. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
  6. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 142
  7. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  8. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  9. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  10. 2023AgentSims: An Open-Source Sandbox for Large Language Model Evaluationcited by 86
  11. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  12. 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
  13. 2023T-SciQ: Teaching Multimodal Chain-of-Thought Reasoning via Mixed Large Language Model Signals for Science Question Answeringcited by 59
  14. 2023MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Recordscited by 54
  15. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
  16. 2024A Survey on LLM-based Multi-Agent System: Recent Advances and New Frontiers in Applicationcited by 54
  17. 2023Lyfe Agents: Generative agents for low-cost real-time social interactionscited by 49
  18. 2024Rethinking Bradley-Terry Models in Preference-Based Reward Modeling: Foundations, Theory, and Alternativescited by 48
  19. 2024ALI-Agent: Assessing LLMs' Alignment with Human Values via Agent-based Evaluationcited by 46
  20. 2025TwinMarket: A Scalable Behavioral and Social Simulation for Financial Marketscited by 46
  21. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 45
  22. 2023Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decodingcited by 40
  23. 2024Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacycited by 36
  24. 2023Trapping LLM Hallucinations Using Tagged Context Promptscited by 34
  25. 2023Large Language Models as Zero-Shot Human Models for Human-Robot Interactioncited by 30
  26. 2024UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulationcited by 30
  27. 2024"My agent understands me better": Integrating Dynamic Human-like Memory Recall and Consolidation in LLM-Based Agentscited by 29
  28. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  29. 2023Wireless Multi-Agent Generative AI: From Connected Intelligence to Collective Intelligencecited by 25
  30. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architecturescited by 25
  31. 2023PromptMix: A Class Boundary Augmentation Method for Large Language Model Distillationcited by 25
  32. 2023SurrealDriver: Designing LLM-powered Generative Driver Agent Framework based on Human Drivers' Driving-thinking Datacited by 24
  33. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 24
  34. 2024Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Taskscited by 23
  35. 2018Relational Forward Models for Multi-Agent Learningcited by 21
  36. 2024StateAct: Enhancing LLM Base Agents via Self-prompting and State-trackingcited by 21
  37. 2024TrustScore: Reference-Free Evaluation of LLM Response Trustworthinesscited by 19
  38. 2024Assistive Large Language Model Agents for Socially-Aware Negotiation Dialoguescited by 18

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026TerraLingua: Emergence and Analysis of Open-endedness in LLM Ecologies
  2. 2026AI Agents Alone Are Not (Yet) Sufficient for Social Simulation
  3. 2026Emergence World: A Platform for Evaluating Long-Horizon Multi-Agent Autonomy
  4. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Design
  5. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  6. 2026Scaling Behavior of Single LLM-Driven Multi-Agent Systems
  7. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
  8. 2025Evaluating Cooperative Resilience in Multiagent Systems: A Comparison Between Humans and LLMs ↗