# Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.06760 · HTML: https://mlsanity.com/paper/2205.06760/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 146 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [RL2: Fast Reinforcement Learning via Slow Reinforcement Learning](https://mlsanity.com/paper/1611.02779/) (2016)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker](https://mlsanity.com/paper/1701.01724/) (2017)
- [Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions](https://mlsanity.com/paper/1901.01753/) (2019)
- [A Study on Overfitting in Deep Reinforcement Learning](https://mlsanity.com/paper/1804.06893/) (2018)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies](https://mlsanity.com/paper/2004.13332/) (2020)
- [Offline Learning from Demonstrations and Unlabeled Experience](https://mlsanity.com/paper/2011.13885/) (2020)
- [Asymmetric self-play for automatic goal discovery in robotic manipulation](https://mlsanity.com/paper/2101.04882/) (2021)
- [Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research](https://mlsanity.com/paper/1903.00742/) (2019)
- [Increasing Generality in Machine Learning through Procedural Content Generation](https://mlsanity.com/paper/1911.13071/) (2019)
- [Imitating Interactive Intelligence](https://mlsanity.com/paper/2012.05672/) (2020)
- [Semantic Exploration from Language Abstractions and Pretrained Representations](https://mlsanity.com/paper/2204.05080/) (2022)
- [Improving Intrinsic Exploration with Language Abstractions](https://mlsanity.com/paper/2202.08938/) (2022)
- [Maintaining cooperation in complex social dilemmas using deep reinforcement learning](https://mlsanity.com/paper/1707.01068/) (2017)
- [ELLA: Exploration through Learned Language Abstraction](https://mlsanity.com/paper/2103.05825/) (2021)
- [Learning Reciprocity in Complex Sequential Social Dilemmas](https://mlsanity.com/paper/1903.08082/) (2019)
- [A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings](https://mlsanity.com/paper/2106.09012/) (2021)
- [Abstraction for Deep Reinforcement Learning](https://mlsanity.com/paper/2202.05839/) (2022)
- [Unsupervised Real-Time Control through Variational Empowerment](https://mlsanity.com/paper/1710.05101/) (2017)
- [Exploring Zero-Shot Emergent Communication in Embodied Multi-Agent Populations](https://mlsanity.com/paper/2010.15896/) (2020)
- [D3C: Reducing the Price of Anarchy in Multi-Agent Learning](https://mlsanity.com/paper/2010.00575/) (2020)
- [Model-free conventions in multi-agent reinforcement learning with heterogeneous preferences](https://mlsanity.com/paper/2010.09054/) (2020)
- [Innateness, AlphaZero, and Artificial Intelligence](https://mlsanity.com/paper/1801.05667/) (2018)
- [Language is Power: Representing States Using Natural Language in Reinforcement Learning](https://mlsanity.com/paper/1910.02789/) (2019)
- [Normative Disagreement as a Challenge for Cooperative AI](https://mlsanity.com/paper/2111.13872/) (2021)

## Built on by (most cited first, 19 of 19)

- [Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia](https://mlsanity.com/paper/2312.03664/) (2023)
- [Open-Endedness is Essential for Artificial Superhuman Intelligence](https://mlsanity.com/paper/2406.04268/) (2024)
- [Melting Pot 2.0](https://mlsanity.com/paper/2211.13746/) (2022)
- [A Review of Cooperation in Multi-agent Learning](https://mlsanity.com/paper/2312.05162/) (2023)
- [Virtual Agent Economies](https://mlsanity.com/paper/2509.10147/) (2025)
- [A social path to human-like artificial intelligence](https://mlsanity.com/paper/2405.15815/) (2024)
- [JaxLife: An Open-Ended Agentic Simulator](https://mlsanity.com/paper/2409.00853/) (2024)
- [Emergent collective intelligence from massive-agent cooperation and competition](https://mlsanity.com/paper/2301.01609/) (2023)
- [Towards Principled Unsupervised Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2502.08365/) (2025)
- [From AGI to ASI](https://mlsanity.com/paper/2606.12683/) (2026)
- [Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysis](https://mlsanity.com/paper/2206.09046/) (2022)
- [Selective Reincarnation: Offline-to-Online Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.00977/) (2023)
- [Understanding the World to Solve Social Dilemmas Using Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.11358/) (2023)
- [Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model](https://mlsanity.com/paper/2411.17724/) (2024)
- [EconGym: A Scalable AI Testbed with Diverse Economic Tasks](https://mlsanity.com/paper/2506.12110/) (2025)
- [Think, Speak, Decide: Language-Augmented Multi-Agent Reinforcement Learning for Economic Decision-Making](https://mlsanity.com/paper/2511.12876/) (2025)
- [Responsible Emergent Multi-Agent Behavior](https://mlsanity.com/paper/2311.01609/) (2023)
- [Solipsistic Superintelligence is Unlikely to be Cooperative](https://mlsanity.com/paper/2606.03237/) (2026)
- [The Foreign Policy AI Evaluation Gap](https://mlsanity.com/paper/2607.02955/) (2026)

## Abstract (arXiv)

Advances in artificial intelligence often stem from the development of new environments that abstract real-world situations into a form where research can be done conveniently. This paper contributes such an environment based on ideas inspired by elementary Microeconomics. Agents learn to produce resources in a spatially complex world, trade them with one another, and consume those that they prefer. We show that the emergent production, consumption, and pricing behaviors respond to environmental conditions in the directions predicted by supply and demand shifts in Microeconomics. We also demonstrate settings where the agents' emergent prices for goods vary over space, reflecting the local abundance of goods. After the price disparities emerge, some agents then discover a niche of transporting goods between regions with different prevailing prices -- a profitable strategy because they can buy goods where they are cheap and sell them where they are expensive. Finally, in a series of ablation experiments, we investigate how choices in the environmental rewards, bartering actions, agent architecture, and ability to consume tradable goods can either aid or inhibit the emergence of this economic behavior. This work is part of the environment development branch of a research program that aims to build human-like artificial general intelligence through multi-agent interactions in simulated societies. By exploring which environment features are needed for the basic phenomena of elementary microeconomics to emerge automatically from learning, we arrive at an environment that differs from those studied in prior multi-agent reinforcement learning work along several dimensions. For example, the model incorporates heterogeneous tastes and physical abilities, and agents negotiate with one another as a grounded form of communication.