# Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set

arXiv: https://arxiv.org/abs/2105.12893 · HTML: https://mlsanity.com/paper/2105.12893/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 128 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [Distributionally Robust Stochastic Optimization with Wasserstein Distance](https://mlsanity.com/paper/1604.02199/) (2016)
- [Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS](https://mlsanity.com/paper/1912.06059/) (2019)
- [Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques](https://mlsanity.com/paper/1605.04965/) (2016)
- [Learning To Simulate](https://mlsanity.com/paper/1810.02513/) (2018)
- [Get Real: Realism Metrics for Robust Limit Order Book Market Simulations](https://mlsanity.com/paper/1912.04941/) (2019)
- [Robust Sample Average Approximation](https://mlsanity.com/paper/1408.4445/) (2014)
- [MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification](https://mlsanity.com/paper/1612.08879/) (2016)
- [A theoretical framework for calibration in computer models: parametrization, estimation and convergence properties](https://mlsanity.com/paper/1508.07155/) (2015)
- [Explaining Agent-Based Financial Market Simulation](https://mlsanity.com/paper/1909.11650/) (2019)
- [Efficient Calibration for Imperfect Computer Models](https://mlsanity.com/paper/1507.07280/) (2015)
- [Image-based model parameter optimization using Model-Assisted Generative Adversarial Networks](https://mlsanity.com/paper/1812.00879/) (2018)
- [Sensitivity to Serial Dependency of Input Processes: A Robust Approach](https://mlsanity.com/paper/1606.06645/) (2016)
- [Subsampling to Enhance Efficiency in Input Uncertainty Quantification](https://mlsanity.com/paper/1811.04500/) (2018)
- [A Frequentist Approach to Computer Model Calibration](https://mlsanity.com/paper/1411.4723/) (2014)
- [Simulator Calibration under Covariate Shift with Kernels](https://mlsanity.com/paper/1809.08159/) (2018)

## Built on by (most cited first, 1 of 1)

- [SOCRATES: Simulation Optimization with Correlated Replicas and Adaptive Trajectory Evaluations](https://mlsanity.com/paper/2511.00685/) (2025)

## Abstract (arXiv)

Stochastic simulation aims to compute output performance for complex models that lack analytical tractability. To ensure accurate prediction, the model needs to be calibrated and validated against real data. Conventional methods approach these tasks by assessing the model-data match via simple hypothesis tests or distance minimization in an ad hoc fashion, but they can encounter challenges arising from non-identifiability and high dimensionality. In this paper, we investigate a framework to develop calibration schemes that satisfy rigorous frequentist statistical guarantees, via a basic notion that we call eligibility set designed to bypass non-identifiability via a set-based estimation. We investigate a feature extraction-then-aggregation approach to construct these sets that target at multivariate outputs. We demonstrate our methodology on several numerical examples, including an application to calibration of a limit order book market simulator (ABIDES).