# A model aggregation approach for high-dimensional large-scale optimization

arXiv: https://arxiv.org/abs/2205.07525 · HTML: https://mlsanity.com/paper/2205.07525/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 75 references that are in the map.
- First posted on arXiv 16 May 2022.

## Builds on (most cited first, 29 of 29)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Scalable Bayesian Optimization Using Deep Neural Networks](https://mlsanity.com/paper/1502.05700/) (2015)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces](https://mlsanity.com/paper/2103.00349/) (2021)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [Exact Gaussian Processes on a Million Data Points](https://mlsanity.com/paper/1903.08114/) (2019)
- [Batched Large-scale Bayesian Optimization in High-dimensional Spaces](https://mlsanity.com/paper/1706.01445/) (2017)
- [Batch Bayesian Optimization via Local Penalization](https://mlsanity.com/paper/1505.08052/) (2015)
- [Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2001.11659/) (2020)
- [A survey on high-dimensional Gaussian process modeling with application to Bayesian optimization](https://mlsanity.com/paper/2111.05040/) (2021)
- [BOCK : Bayesian Optimization with Cylindrical Kernels](https://mlsanity.com/paper/1806.01619/) (2018)
- [Local Gaussian process approximation for large computer experiments](https://mlsanity.com/paper/1303.0383/) (2013)
- [Parallel Bayesian Global Optimization of Expensive Functions](https://mlsanity.com/paper/1602.05149/) (2016)
- [High Dimensional Bayesian Optimization Using Dropout](https://mlsanity.com/paper/1802.05400/) (2018)
- [Bayesian model averaging: A systematic review and conceptual classification](https://mlsanity.com/paper/1509.08864/) (2015)
- [On the choice of the low-dimensional domain for global optimization via random embeddings](https://mlsanity.com/paper/1704.05318/) (2017)
- [Nested Kriging predictions for datasets with large number of observations](https://mlsanity.com/paper/1607.05432/) (2016)
- [A warped kernel improving robustness in Bayesian optimization via random embeddings](https://mlsanity.com/paper/1411.3685/) (2014)
- [Differentiating the multipoint Expected Improvement for optimal batch design](https://mlsanity.com/paper/1503.05509/) (2015)
- [Spatially Aggregated Gaussian Processes with Multivariate Areal Outputs](https://mlsanity.com/paper/1907.08350/) (2019)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [A dimensionality reduction technique for unconstrained global optimization of functions with low effective dimensionality](https://mlsanity.com/paper/2003.09673/) (2020)
- [Surrogate-Based Simulation Optimization](https://mlsanity.com/paper/2105.03893/) (2021)
- [Heteroscedastic Bayesian Optimisation for Stochastic Model Predictive Control](https://mlsanity.com/paper/2010.00202/) (2020)
- [Constrained global optimization of functions with low effective dimensionality using multiple random embeddings](https://mlsanity.com/paper/2009.10446/) (2020)
- [Combined Global and Local Search for Optimization with Gaussian Process Models](https://mlsanity.com/paper/2107.03217/) (2021)
- [A Multi-Level Simulation Optimization Approach for Quantile Functions](https://mlsanity.com/paper/1901.05768/) (2019)
- [Precision Aggregated Local Models](https://mlsanity.com/paper/2005.13375/) (2020)

## Built on by (most cited first, 1 of 1)

- [Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions](https://mlsanity.com/paper/2411.16206/) (2024)

## Abstract (arXiv)

Bayesian optimization (BO) has been widely used in machine learning and simulation optimization. With the increase in computational resources and storage capacities in these fields, high-dimensional and large-scale problems are becoming increasingly common. In this study, we propose a model aggregation method in the Bayesian optimization (MamBO) algorithm for efficiently solving high-dimensional large-scale optimization problems. MamBO uses a combination of subsampling and subspace embeddings to collectively address high dimensionality and large-scale issues; in addition, a model aggregation method is employed to address the surrogate model uncertainty issue that arises when embedding is applied. This surrogate model uncertainty issue is largely ignored in the embedding literature and practice, and it is exacerbated when the problem is high-dimensional and data are limited. Our proposed model aggregation method reduces these lower-dimensional surrogate model risks and improves the robustness of the BO algorithm. We derive an asymptotic bound for the proposed aggregated surrogate model and prove the convergence of MamBO. Benchmark numerical experiments indicate that our algorithm achieves superior or comparable performance to other commonly used high-dimensional BO algorithms. Moreover, we apply MamBO to a cascade classifier of a machine learning algorithm for face detection, and the results reveal that MamBO finds settings that achieve higher classification accuracy than the benchmark settings and is computationally faster than other high-dimensional BO algorithms.