# A multi-agent reinforcement learning model of common-pool resource appropriation

arXiv: https://arxiv.org/abs/1707.06600 · HTML: https://mlsanity.com/paper/1707.06600/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jul 2017.

## Built on by (most cited first, 30 of 69)

- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents](https://mlsanity.com/paper/2404.16698/) (2024)
- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Melting Pot 2.0](https://mlsanity.com/paper/2211.13746/) (2022)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [Collective eXplainable AI: Explaining Cooperative Strategies and Agent Contribution in Multiagent Reinforcement Learning with Shapley Values](https://mlsanity.com/paper/2110.01307/) (2021)
- [Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems](https://mlsanity.com/paper/2102.07659/) (2021)
- [Quantifying the effects of environment and population diversity in multi-agent reinforcement learning](https://mlsanity.com/paper/2102.08370/) (2021)
- [Virtual Agent Economies](https://mlsanity.com/paper/2509.10147/) (2025)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Learning and Sustaining Shared Normative Systems via Bayesian Rule Induction in Markov Games](https://mlsanity.com/paper/2402.13399/) (2024)
- [MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2506.14990/) (2025)
- [Cooperative Resilience in Artificial Intelligence Multiagent Systems](https://mlsanity.com/paper/2409.13187/) (2024)
- [Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL](https://mlsanity.com/paper/2104.11455/) (2021)
- [Hidden Agenda: a Social Deduction Game with Diverse Learned Equilibria](https://mlsanity.com/paper/2201.01816/) (2022)
- [A multi-agent reinforcement learning model of reputation and cooperation in human groups](https://mlsanity.com/paper/2103.04982/) (2021)
- [JaxLife: An Open-Ended Agentic Simulator](https://mlsanity.com/paper/2409.00853/) (2024)
- [Deep Reinforcement Learning for Conservation Decisions](https://mlsanity.com/paper/2106.08272/) (2021)
- [Learning to Share in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2112.08702/) (2021)
- [Investigating the Impact of Direct Punishment on the Emergence of Cooperation in Multi-Agent Reinforcement Learning Systems](https://mlsanity.com/paper/2301.08278/) (2023)
- [Mediated Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.08419/) (2023)
- [Predicting Multi-Agent Specialization via Task Parallelizability](https://mlsanity.com/paper/2503.15703/) (2025)
- [Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective](https://mlsanity.com/paper/2106.05402/) (2021)
- [Statistical discrimination in learning agents](https://mlsanity.com/paper/2110.11404/) (2021)
- [Co-Learning Empirical Games and World Models](https://mlsanity.com/paper/2305.14223/) (2023)
- [Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation](https://mlsanity.com/paper/2405.19328/) (2024)
- [Improved Cooperation by Exploiting a Common Signal](https://mlsanity.com/paper/2102.02304/) (2021)
- [Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environments](https://mlsanity.com/paper/2102.12307/) (2021)
- [Improving Social Welfare While Preserving Autonomy via a Pareto Mediator](https://mlsanity.com/paper/2106.03927/) (2021)

## Abstract (arXiv)

Humanity faces numerous problems of common-pool resource appropriation. This class of multi-agent social dilemma includes the problems of ensuring sustainable use of fresh water, common fisheries, grazing pastures, and irrigation systems. Abstract models of common-pool resource appropriation based on non-cooperative game theory predict that self-interested agents will generally fail to find socially positive equilibria---a phenomenon called the tragedy of the commons. However, in reality, human societies are sometimes able to discover and implement stable cooperative solutions. Decades of behavioral game theory research have sought to uncover aspects of human behavior that make this possible. Most of that work was based on laboratory experiments where participants only make a single choice: how much to appropriate. Recognizing the importance of spatial and temporal resource dynamics, a recent trend has been toward experiments in more complex real-time video game-like environments. However, standard methods of non-cooperative game theory can no longer be used to generate predictions for this case. Here we show that deep reinforcement learning can be used instead. To that end, we study the emergent behavior of groups of independently learning agents in a partially observed Markov game modeling common-pool resource appropriation. Our experiments highlight the importance of trial-and-error learning in common-pool resource appropriation and shed light on the relationship between exclusion, sustainability, and inequality.