# Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling

arXiv: https://arxiv.org/abs/2405.02161 · HTML: https://mlsanity.com/paper/2405.02161/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 48 references that are in the map.
- First posted on arXiv 3 May 2024.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 11 of 11)

- [Reinforcement Learning in Economics and Finance](https://mlsanity.com/paper/2003.10014/) (2020)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [Learning to simulate realistic limit order book markets from data as a World Agent](https://mlsanity.com/paper/2210.09897/) (2022)
- [On learning agent-based models from data](https://mlsanity.com/paper/2205.05052/) (2022)
- [Solving Heterogeneous General Equilibrium Economic Models with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.16977/) (2021)
- [Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations](https://mlsanity.com/paper/2210.07184/) (2022)
- [TaxAI: A Dynamic Economic Simulator and Benchmark for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2309.16307/) (2023)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs](https://mlsanity.com/paper/2302.11835/) (2023)
- [An Economy of Neural Networks: Learning from Heterogeneous Experiences](https://mlsanity.com/paper/2110.11582/) (2021)
- [A Heterogeneous Agent Model of Mortgage Servicing: An Income-based Relief Analysis](https://mlsanity.com/paper/2402.17932/) (2024)

## Built on by (most cited first, 5 of 5)

- [EconGym: A Scalable AI Testbed with Diverse Economic Tasks](https://mlsanity.com/paper/2506.12110/) (2025)
- [Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.12272/) (2025)
- [PersonaLedger: Generating Realistic Financial Transactions with Persona Conditioned LLMs and Rule Grounded Feedback](https://mlsanity.com/paper/2601.03149/) (2026)
- [From Individual Learning to Market Equilibrium: Correcting Structural and Parametric Biases in RL Simulations of Economic Models](https://mlsanity.com/paper/2507.18229/) (2025)
- [Natural-gas storage modelling by deep reinforcement learning](https://mlsanity.com/paper/2511.02646/) (2025)

## Abstract (arXiv)

Agent-based models (ABMs) are simulation models used in economics to overcome some of the limitations of traditional frameworks based on general equilibrium assumptions. However, agents within an ABM follow predetermined 'bounded rational' behavioural rules which can be cumbersome to design and difficult to justify. Here we leverage multi-agent reinforcement learning (RL) to expand the capabilities of ABMs with the introduction of 'fully rational' agents that learn their policy by interacting with the environment and maximising a reward function. Specifically, we propose a 'Rational macro ABM' (R-MABM) framework by extending a paradigmatic macro ABM from the economic literature. We show that gradually substituting ABM firms in the model with RL agents, trained to maximise profits, allows for studying the impact of rationality on the economy. We find that RL agents spontaneously learn three distinct strategies for maximising profits, with the optimal strategy depending on the level of market competition and rationality. We also find that RL agents with independent policies, and without the ability to communicate with each other, spontaneously learn to segregate into different strategic groups, thus increasing market power and overall profits. Finally, we find that a higher number of rational (RL) agents in the economy always improves the macroeconomic environment as measured by total output. Depending on the specific rational policy, this can come at the cost of higher instability. Our R-MABM framework allows for stable multi-agent learning, is available in open source, and represents a principled and robust direction to extend economic simulators.