# Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions

arXiv: https://arxiv.org/abs/2504.20527 · HTML: https://mlsanity.com/paper/2504.20527/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 71 references that are in the map.
- First posted on arXiv 29 Apr 2025.

## Builds on (most cited first, 28 of 28)

- [Variational Quantum Algorithms](https://mlsanity.com/paper/2012.09265/) (2020)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets](https://mlsanity.com/paper/1605.07079/) (2016)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Constrained Bayesian Optimization with Noisy Experiments](https://mlsanity.com/paper/1706.07094/) (2017)
- [HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation](https://mlsanity.com/paper/2012.03826/) (2020)
- [A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization](https://mlsanity.com/paper/2111.05040/) (2021)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [MCMC for Variationally Sparse Gaussian Processes](https://mlsanity.com/paper/1506.04000/) (2015)
- [Practical heteroskedastic Gaussian process modeling for large simulation experiments](https://mlsanity.com/paper/1611.05902/) (2016)
- [GLASSES: Relieving The Myopia Of Bayesian Optimisation](https://mlsanity.com/paper/1510.06299/) (2015)
- [Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation](https://mlsanity.com/paper/2301.10123/) (2023)
- [Replication or exploration? Sequential design for stochastic simulation experiments](https://mlsanity.com/paper/1710.03206/) (2017)
- [Local Bayesian optimization via maximizing probability of descent](https://mlsanity.com/paper/2210.11662/) (2022)
- [TREGO: a Trust-Region Framework for Efficient Global Optimization](https://mlsanity.com/paper/2101.06808/) (2021)
- [Optimization, fast and slow: optimally switching between local and Bayesian optimization](https://mlsanity.com/paper/1805.08610/) (2018)
- [The Behavior and Convergence of Local Bayesian Optimization](https://mlsanity.com/paper/2305.15572/) (2023)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [Differentiating the multipoint Expected Improvement for optimal batch design](https://mlsanity.com/paper/1503.05509/) (2015)
- [Evaluating Gaussian Process Metamodels and Sequential Designs for Noisy Level Set Estimation](https://mlsanity.com/paper/1807.06712/) (2018)
- [A portfolio approach to massively parallel Bayesian optimization](https://mlsanity.com/paper/2110.09334/) (2021)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [ASTRO-DF: A Class of Adaptive Sampling Trust-Region Algorithms for Derivative-Free Stochastic Optimization](https://mlsanity.com/paper/1610.06506/) (2016)
- [Movement Penalized Bayesian Optimization with Application to Wind Energy Systems](https://mlsanity.com/paper/2210.08087/) (2022)
- [A trust-region method for derivative-free nonlinear constrained stochastic optimization](https://mlsanity.com/paper/1703.04156/) (2017)
- [Accelerating Noisy VQE Optimization with Gaussian Processes](https://mlsanity.com/paper/2204.07331/) (2022)
- [A Novel Noise-Aware Classical Optimizer for Variational Quantum Algorithms](https://mlsanity.com/paper/2401.10121/) (2024)

## Abstract (arXiv)

We develop and analyze a method for stochastic simulation optimization based on Gaussian process models within a trust-region framework. We focus on settings where the variance of the objective function is large, making accurate estimation challenging and often requiring many evaluations. To address this regime, we combine local modeling with adaptive replication, allowing the method to allocate repeated evaluations where they are most beneficial. We introduce several mechanisms to promote and adapt replication, including modifications to the acquisition function and cost-aware evaluation strategies. These components enable our approach to scale effectively when high levels of sampling are required to reduce noise. We refer to the resulting method as OGPIT, for Optimization by Gaussian Processes In Trust regions. Numerical experiments show that adaptive replication can substantially improve computational efficiency while preserving solution accuracy compared to baseline methods, in particular when evaluation costs are taken into account.