# Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2510.12272 · HTML: https://mlsanity.com/paper/2510.12272/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 54 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Markets](https://mlsanity.com/paper/2110.14771/) (2021)
- [Deep Reinforcement Learning in a Monetary Model](https://mlsanity.com/paper/2104.09368/) (2021)
- [SuperSuit: Simple Microwrappers for Reinforcement Learning Environments](https://mlsanity.com/paper/2008.08932/) (2020)
- [Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.16977/) (2021)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs](https://mlsanity.com/paper/2302.11835/) (2023)
- [Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling](https://mlsanity.com/paper/2405.02161/) (2024)
- [Reinforcement Learning for Economic Policy: A New Frontier?](https://mlsanity.com/paper/2206.08781/) (2022)
- [Mean-Field Games With Finitely Many Players: Independent Learning and Subjectivity](https://mlsanity.com/paper/2209.05703/) (2022)
- [ADAGE: A generic two-layer framework for adaptive agent based modelling](https://mlsanity.com/paper/2501.09429/) (2025)
- [Tax Credits and Household Behavior: The Roles of Myopic Decision-Making and Liquidity in a Simulated Economy](https://mlsanity.com/paper/2408.10391/) (2024)

## Built on by (most cited first, 1 of 1)

- [Structural Reinforcement Learning for Heterogeneous Agent Macroeconomics](https://mlsanity.com/paper/2512.18892/) (2025)

## Abstract (arXiv)

Current macroeconomic models with agent heterogeneity can be broadly divided into two main groups. Heterogeneous-agent general equilibrium (GE) models, such as those based on Heterogeneous Agent New Keynesian (HANK) or Krusell-Smith (KS) approaches, rely on GE and 'rational expectations', somewhat unrealistic assumptions that make the models very computationally cumbersome, which in turn limits the amount of heterogeneity that can be modelled. In contrast, agent-based models (ABMs) can flexibly encompass a large number of arbitrarily heterogeneous agents, but typically require the specification of explicit behavioural rules, which can lead to a lengthy trial-and-error model-development process. To address these limitations, we introduce MARL-BC, a framework that integrates deep multi-agent reinforcement learning (MARL) with real business cycle (RBC) models. We demonstrate that MARL-BC can: (1) recover textbook RBC results when using a single agent; (2) recover the results of the mean-field KS model using a large number of identical agents; and (3) effectively simulate rich heterogeneity among agents, a hard task for traditional GE approaches. Our framework can be thought of as an ABM if used with a variety of heterogeneous interacting agents, and can reproduce GE results in limit cases. As such, it is a step towards a synthesis of these often opposed modelling paradigms.