# Efficient Calibration of Multi-Agent Simulation Models from Output Series with Bayesian Optimization

arXiv: https://arxiv.org/abs/2112.03874 · HTML: https://mlsanity.com/paper/2112.03874/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 6 of 6)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [Explaining Agent-Based Financial Market Simulation](https://mlsanity.com/paper/1909.11650/) (2019)
- [Learning who is in the market from time series: market participant discovery through adversarial calibration of multi-agent simulators](https://mlsanity.com/paper/2108.00664/) (2021)

## Built on by (most cited first, 5 of 5)

- [Conditional Generators for Limit Order Book Environments: Explainability, Challenges, and Robustness](https://mlsanity.com/paper/2306.12806/) (2023)
- [Pseudo-Bayesian Optimization](https://mlsanity.com/paper/2310.09766/) (2023)
- [Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks](https://mlsanity.com/paper/2311.11913/) (2023)
- [Bias and Identifiability in the Bounded Confidence Model](https://mlsanity.com/paper/2506.11751/) (2025)
- [Online Regime-aware Calibration for Black-box Social Simulators via Posterior-assisted Evolutionary Dynamic Optimization](https://mlsanity.com/paper/2601.19481/) (2026)

## Abstract (arXiv)

Multi-agent simulation is commonly used across multiple disciplines, specifically in artificial intelligence in recent years, which creates an environment for downstream machine learning or reinforcement learning tasks. In many practical scenarios, however, only the output series that result from the interactions of simulation agents are observable. Therefore, simulators need to be calibrated so that the simulated output series resemble historical -- which amounts to solving a complex simulation optimization problem. In this paper, we propose a simple and efficient framework for calibrating simulator parameters from historical output series observations. First, we consider a novel concept of eligibility set to bypass the potential non-identifiability issue. Second, we generalize the two-sample Kolmogorov-Smirnov (K-S) test with Bonferroni correction to test the similarity between two high-dimensional distributions, which gives a simple yet effective distance metric between the output series sample sets. Third, we suggest using Bayesian optimization (BO) and trust-region BO (TuRBO) to minimize the aforementioned distance metric. Finally, we demonstrate the efficiency of our framework using numerical experiments both on a multi-agent financial market simulator.