# Multi-fidelity Constrained Optimization for Stochastic Black Box Simulators

arXiv: https://arxiv.org/abs/2311.15137 · HTML: https://mlsanity.com/paper/2311.15137/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 25 Nov 2023.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Survey of multifidelity methods in uncertainty propagation, inference, and optimization](https://mlsanity.com/paper/1806.10761/) (2018)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Learning To Simulate](https://mlsanity.com/paper/1810.02513/) (2018)
- [Adversarial Variational Optimization of Non-Differentiable Simulators](https://mlsanity.com/paper/1707.07113/) (2017)
- [A probabilistic, data-driven closure model for RANS simulations with aleatoric, model uncertainty](https://mlsanity.com/paper/2307.02432/) (2023)
- [Stochastic Variational Optimization](https://mlsanity.com/paper/1809.04855/) (2018)
- [A trust-region method for derivative-free nonlinear constrained stochastic optimization](https://mlsanity.com/paper/1703.04156/) (2017)

## Built on by (most cited first, 3 of 3)

- [Multi-fidelity Gaussian process surrogate modeling for regression problems in physics](https://mlsanity.com/paper/2404.11965/) (2024)
- [From concrete mixture to structural design -- a holistic optimization procedure in the presence of uncertainties](https://mlsanity.com/paper/2312.03607/) (2023)
- [Simulating, Fast and Slow: Learning Policies for Black-Box Optimization](https://mlsanity.com/paper/2406.04261/) (2024)

## Abstract (arXiv)

Constrained optimization of the parameters of a simulator plays a crucial role in a design process. These problems become challenging when the simulator is stochastic, computationally expensive, and the parameter space is high-dimensional. One can efficiently perform optimization only by utilizing the gradient with respect to the parameters, but these gradients are unavailable in many legacy, black-box codes. We introduce the algorithm Scout-Nd (Stochastic Constrained Optimization for N dimensions) to tackle the issues mentioned earlier by efficiently estimating the gradient, reducing the noise of the gradient estimator, and applying multi-fidelity schemes to further reduce computational effort. We validate our approach on standard benchmarks, demonstrating its effectiveness in optimizing parameters highlighting better performance compared to existing methods.