# Malthusian Reinforcement Learning

arXiv: https://arxiv.org/abs/1812.07019 · HTML: https://mlsanity.com/paper/1812.07019/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2018.

## Built on by (most cited first, 14 of 14)

- [Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot](https://mlsanity.com/paper/2107.06857/) (2021)
- [Melting Pot 2.0](https://mlsanity.com/paper/2211.13746/) (2022)
- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [Quantifying the effects of environment and population diversity in multi-agent reinforcement learning](https://mlsanity.com/paper/2102.08370/) (2021)
- [A social path to human-like artificial intelligence](https://mlsanity.com/paper/2405.15815/) (2024)
- [HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging](https://mlsanity.com/paper/2102.00824/) (2021)
- [Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments](https://mlsanity.com/paper/2209.10958/) (2022)
- [Diversity Through Exclusion (DTE): Niche Identification for Reinforcement Learning through Value-Decomposition](https://mlsanity.com/paper/2302.01180/) (2023)
- [Fairness Aware Reinforcement Learning via Proximal Policy Optimization](https://mlsanity.com/paper/2502.03953/) (2025)
- [When Is Diversity Rewarded in Cooperative Multi-Agent Learning?](https://mlsanity.com/paper/2506.09434/) (2025)
- [AutoGen Driven Multi Agent Framework for Iterative Crime Data Analysis and Prediction](https://mlsanity.com/paper/2506.11475/) (2025)
- [The Emergence of Complex Behavior in Large-Scale Ecological Environments](https://mlsanity.com/paper/2510.18221/) (2025)
- [Replication of Multi-agent Reinforcement Learning for the "Hide and Seek" Problem](https://mlsanity.com/paper/2310.05430/) (2023)
- [Grounded Scaling: Why Agentic AI Needs Deterministic Environments](https://mlsanity.com/paper/2606.22495/) (2026)

## Abstract (arXiv)

Here we explore a new algorithmic framework for multi-agent reinforcement learning, called Malthusian reinforcement learning, which extends self-play to include fitness-linked population size dynamics that drive ongoing innovation. In Malthusian RL, increases in a subpopulation's average return drive subsequent increases in its size, just as Thomas Malthus argued in 1798 was the relationship between preindustrial income levels and population growth. Malthusian reinforcement learning harnesses the competitive pressures arising from growing and shrinking population size to drive agents to explore regions of state and policy spaces that they could not otherwise reach. Furthermore, in environments where there are potential gains from specialization and division of labor, we show that Malthusian reinforcement learning is better positioned to take advantage of such synergies than algorithms based on self-play.