# Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.16977 · HTML: https://mlsanity.com/paper/2103.16977/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Revisiting Fundamentals of Experience Replay](https://mlsanity.com/paper/2007.06700/) (2020)
- [Learning Continuous Control Policies by Stochastic Value Gradients](https://mlsanity.com/paper/1510.09142/) (2015)
- [The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies](https://mlsanity.com/paper/2004.13332/) (2020)
- [Compatible Value Gradients for Reinforcement Learning of Continuous Deep Policies](https://mlsanity.com/paper/1509.03005/) (2015)

## Built on by (most cited first, 10 of 10)

- [PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2308.02585/) (2023)
- [Contextual Bilevel Reinforcement Learning for Incentive Alignment](https://mlsanity.com/paper/2406.01575/) (2024)
- [Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling](https://mlsanity.com/paper/2405.02161/) (2024)
- [Can an AI agent hit a moving target?](https://mlsanity.com/paper/2110.02474/) (2021)
- [EconGym: A Scalable AI Testbed with Diverse Economic Tasks](https://mlsanity.com/paper/2506.12110/) (2025)
- [Analyzing Micro-Founded General Equilibrium Models with Many Agents using Deep Reinforcement Learning](https://mlsanity.com/paper/2201.01163/) (2022)
- [Empirical Equilibria in Agent-based Economic systems with Learning agents](https://mlsanity.com/paper/2408.12038/) (2024)
- [Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.12272/) (2025)
- [Learning Macroeconomic Policies through Dynamic Stackelberg Mean-Field Games](https://mlsanity.com/paper/2403.12093/) (2024)
- [Large-Scale Contextual Market Equilibrium Computation through Deep Learning](https://mlsanity.com/paper/2406.15459/) (2024)

## Abstract (arXiv)

General equilibrium macroeconomic models are a core tool used by policymakers to understand a nation's economy. They represent the economy as a collection of forward-looking actors whose behaviours combine, possibly with stochastic effects, to determine global variables (such as prices) in a dynamic equilibrium. However, standard semi-analytical techniques for solving these models make it difficult to include the important effects of heterogeneous economic actors. The COVID-19 pandemic has further highlighted the importance of heterogeneity, for example in age and sector of employment, in macroeconomic outcomes and the need for models that can more easily incorporate it. We use techniques from reinforcement learning to solve such models incorporating heterogeneous agents in a way that is simple, extensible, and computationally efficient. We demonstrate the method's accuracy and stability on a toy problem for which there is a known analytical solution, its versatility by solving a general equilibrium problem that includes global stochasticity, and its flexibility by solving a combined macroeconomic and epidemiological model to explore the economic and health implications of a pandemic. The latter successfully captures plausible economic behaviours induced by differential health risks by age.