# Hyperparameter Adaptive Search for Surrogate Optimization: A Self-Adjusting Approach

arXiv: https://arxiv.org/abs/2310.07970 · HTML: https://mlsanity.com/paper/2310.07970/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 19 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [The Parallel Knowledge Gradient Method for Batch Bayesian Optimization](https://mlsanity.com/paper/1606.04414/) (2016)
- [Financial Applications of Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/1903.04841/) (2019)
- [High-dimensional Black-box Optimization Under Uncertainty](https://mlsanity.com/paper/1911.02457/) (2019)
- [An Empirical Review of Model-based Adaptive Sampling for Global Optimization of Expensive Black-box Functions](https://mlsanity.com/paper/2204.10795/) (2022)
- [Towards Automated Design of Bayesian Optimization via Exploratory Landscape Analysis](https://mlsanity.com/paper/2211.09678/) (2022)

## Built on by (most cited first, 2 of 2)

- [TESO Tabu Enhanced Simulation Optimization for Noisy Black Box Problems](https://mlsanity.com/paper/2512.24007/) (2025)
- [Multi-Agent LLMs for Adaptive Acquisition in Bayesian Optimization](https://mlsanity.com/paper/2603.28959/) (2026)

## Abstract (arXiv)

Surrogate Optimization (SO) algorithms have shown promise for optimizing expensive black-box functions. However, their performance is heavily influenced by hyperparameters related to sampling and surrogate fitting, which poses a challenge to their widespread adoption. We investigate the impact of hyperparameters on various SO algorithms and propose a Hyperparameter Adaptive Search for SO (HASSO) approach. HASSO is not a hyperparameter tuning algorithm, but a generic self-adjusting SO algorithm that dynamically tunes its own hyperparameters while concurrently optimizing the primary objective function, without requiring additional evaluations. The aim is to improve the accessibility, effectiveness, and convergence speed of SO algorithms for practitioners. Our approach identifies and modifies the most influential hyperparameters specific to each problem and SO approach, reducing the need for manual tuning without significantly increasing the computational burden. Experimental results demonstrate the effectiveness of HASSO in enhancing the performance of various SO algorithms across different global optimization test problems.