mlsanityData to 14 Oct 2026

Virtual Agent Economies

arXiv 2509.10147 ↗First posted 12 Sep 2025cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

The rapid adoption of autonomous AI agents is giving rise to a new economic layer where agents transact and coordinate at scales and speeds beyond direct human oversight. We propose the "sandbox economy" as a framework for analyzing this emergent system, characterizing it along two key dimensions: its origins (emergent vs. intentional) and its degree of separateness from the established human economy (permeable vs. impermeable). Our current trajectory points toward a spontaneous emergence of a vast and highly permeable AI agent economy, presenting us with opportunities for an unprecedented degree of coordination as well as significant challenges, including systemic economic risk and exacerbated inequality. Here we discuss a number of possible design choices that may lead to safely steerable AI agent markets. In particular, we consider auction mechanisms for fair resource allocation and preference resolution, the design of AI "mission economies" to coordinate around achieving collective goals, and socio-technical infrastructure needed to ensure trust, safety, and accountability. By doing this, we argue for the proactive design of steerable agent markets to ensure the coming technological shift aligns with humanity's long-term collective flourishing.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 98 of its 271 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

98 of 271 references.

References · 98

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  4. 2024A Survey on LLM-as-a-Judgecited by 1,423
  5. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  6. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  7. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  8. 2020Artificial Intelligence, Values and Alignmentcited by 448
  9. 2018Scalable agent alignment via reward modeling: a research directioncited by 447
  10. 2024Understanding the planning of LLM agents: A surveycited by 421
  11. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  12. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
38 more
  1. 2022The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Modelscited by 328
  2. 2019Measurement and Fairnesscited by 308
  3. 2025Small Language Models are the Future of Agentic AIcited by 270
  4. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 264
  5. 2023AI Alignment: A Comprehensive Surveycited by 263
  6. 2022The Alignment Problem from a Deep Learning Perspectivecited by 258
  7. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  8. 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
  9. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  10. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  11. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
  12. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  13. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  14. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 175
  15. 2025Multi-Agent Risks from Advanced AIcited by 163
  16. 2021Collaborating with Humans without Human Datacited by 142
  17. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  18. 2024Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agentscited by 125
  19. 2017Emergent Complexity via Multi-Agent Competitioncited by 120
  20. 2024The Ethics of Advanced AI Assistantscited by 107
  21. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  22. 2024The rising costs of training frontier AI modelscited by 95
  23. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 93
  24. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 92
  25. 2025Reward Shaping to Mitigate Reward Hacking in RLHFcited by 88
  26. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 87
  27. 2021Consequences of Misaligned AIcited by 80
  28. 2023ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planningcited by 77
  29. 2024RRM: Robust Reward Model Training Mitigates Reward Hackingcited by 76
  30. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 70
  31. 2017A multi-agent reinforcement learning model of common-pool resource appropriationcited by 69
  32. 2018Modeling Others using Oneself in Multi-Agent Reinforcement Learningcited by 66
  33. 2019Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Researchcited by 61
  34. 2024Game-theoretic LLM: Agent Workflow for Negotiation Gamescited by 56
  35. 2023Tree-Planner: Efficient Close-loop Task Planning with Large Language Modelscited by 48
  36. 2019Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Surveycited by 47
  37. 2025Gradual Disempowerment: Systemic Existential Risks from Incremental AI Developmentcited by 46
  38. 2025An Approach to Technical AGI Safety and Securitycited by 45

50 most cited of 98 All 98 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 15.

21 papers.

Sort:

By area:

  1. 2026Institutional AI: Governing LLM Collusion in Multi-Agent Cournot Markets via Public Governance Graphscited by 20
  2. 2025A Pragmatic View of AI Personhoodcited by 12
  3. 2025Distributional AGI Safetycited by 12
  4. 2025Echoing: Identity Failures when LLM Agents Talk to Each Othercited by 9
  5. 2026Social physics in the age of artificial intelligencecited by 5
  6. 2026A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarmscited by 5
  7. 2026From AGI to ASIcited by 4
  8. 2025Agentic Inequalitycited by 3
  9. 2026Scaling Small Agents Through Strategy Auctionscited by 3
  10. 2026Discovering Differences in Strategic Behavior Between Humans and LLMscited by 3
  11. 2026Architecting Trust in Artificial Epistemic Agentscited by 3
  12. 2026AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Powercited by 3
9 more
  1. 2026Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiationcited by 1
  2. 2025Modeling AI-Driven Production and Competitiveness A Multi-Agent Economic Simulation of China and the United Statescited by 0
  3. 2025When AI Agents Compete for Jobs: Strategic Capabilities and Economic Dynamics of AI Labour Marketscited by 0
  4. 2026Governance by Design: A Parsonian Institutional Architecture for Internet-Wide Agent Societiescited by 0
  5. 2026Foundation Protocol: A Coordination Layer for Agentic Societycited by 0
  6. 2026SODE: Analyzing Social Dynamics in LLM Agentscited by 0
  7. 2026The evolution of sex for artificial intelligence: a population-genetic framework for multigenerational model populationscited by 0
  8. 2026SoK: Decentralized Agent Economic Infrastructurecited by 0
  9. 2026The Wealth of Agents: Scaling Agentic AI Across Developerscited by 0