# Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs

arXiv: https://arxiv.org/abs/2302.11835 · HTML: https://mlsanity.com/paper/2302.11835/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 72 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 8 of 8)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Taming Non-stationary Bandits: A Bayesian Approach](https://mlsanity.com/paper/1707.09727/) (2017)
- [On learning agent-based models from data](https://mlsanity.com/paper/2205.05052/) (2022)
- [Black-box Bayesian inference for economic agent-based models](https://mlsanity.com/paper/2202.00625/) (2022)
- [Towards a fully RL-based Market Simulator](https://mlsanity.com/paper/2110.06829/) (2021)
- [Towards Multi-Agent Reinforcement Learning driven Over-The-Counter Market Simulations](https://mlsanity.com/paper/2210.07184/) (2022)
- [Phantom -- A RL-driven multi-agent framework to model complex systems](https://mlsanity.com/paper/2210.06012/) (2022)
- [Blackbox: A procedure for parallel optimization of expensive black-box functions](https://mlsanity.com/paper/1605.00998/) (2016)

## Built on by (most cited first, 5 of 5)

- [Simulating the Economic Impact of Rationality through Reinforcement Learning and Agent-Based Modelling](https://mlsanity.com/paper/2405.02161/) (2024)
- [Learning Individual Behavior in Agent-Based Models with Graph Diffusion Networks](https://mlsanity.com/paper/2505.21426/) (2025)
- [Variational Inference of Parameters in Opinion Dynamics Models](https://mlsanity.com/paper/2403.05358/) (2024)
- [Heterogeneous RBCs via Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2510.12272/) (2025)
- [Natural-gas storage modelling by deep reinforcement learning](https://mlsanity.com/paper/2511.02646/) (2025)

## Abstract (arXiv)

Calibrating agent-based models (ABMs) in economics and finance typically involves a derivative-free search in a very large parameter space. In this work, we benchmark a number of search methods in the calibration of a well-known macroeconomic ABM on real data, and further assess the performance of "mixed strategies" made by combining different methods. We find that methods based on random-forest surrogates are particularly efficient, and that combining search methods generally increases performance since the biases of any single method are mitigated. Moving from these observations, we propose a reinforcement learning (RL) scheme to automatically select and combine search methods on-the-fly during a calibration run. The RL agent keeps exploiting a specific method only as long as this keeps performing well, but explores new strategies when the specific method reaches a performance plateau. The resulting RL search scheme outperforms any other method or method combination tested, and does not rely on any prior information or trial and error procedure.