# TREGO: a Trust-Region Framework for Efficient Global Optimization

arXiv: https://arxiv.org/abs/2101.06808 · HTML: https://mlsanity.com/paper/2101.06808/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 63 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 14 of 14)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Scalable Global Optimization via Local Bayesian Optimization](https://mlsanity.com/paper/1910.01739/) (2019)
- [Bayesian Optimization in a Billion Dimensions via Random Embeddings](https://mlsanity.com/paper/1301.1942/) (2013)
- [High Dimensional Bayesian Optimisation and Bandits via Additive Models](https://mlsanity.com/paper/1503.01673/) (2015)
- [COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting](https://mlsanity.com/paper/1603.08785/) (2016)
- [BOCK : Bayesian Optimization with Cylindrical Kernels](https://mlsanity.com/paper/1806.01619/) (2018)
- [Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian](https://mlsanity.com/paper/1605.09466/) (2016)
- [Stochastic Optimization Using a Trust-Region Method and Random Models](https://mlsanity.com/paper/1504.04231/) (2015)
- [Optimization, fast and slow: optimally switching between local and Bayesian optimization](https://mlsanity.com/paper/1805.08610/) (2018)
- [Correcting boundary over-exploration deficiencies in Bayesian optimization with virtual derivative sign observations](https://mlsanity.com/paper/1704.00963/) (2017)
- [A stochastic Levenberg-Marquardt method using random models with complexity results](https://mlsanity.com/paper/1807.02176/) (2018)
- [A Taxonomy of Constraints in Simulation-Based Optimization](https://mlsanity.com/paper/1505.07881/) (2015)
- [A Merit Function Approach for Evolution Strategies](https://mlsanity.com/paper/1804.11111/) (2018)
- [StoMADS: Stochastic blackbox optimization using probabilistic estimates](https://mlsanity.com/paper/1911.01012/) (2019)

## Built on by (most cited first, 29 of 29)

- [Inducing Point Allocation for Sparse Gaussian Processes in High-Throughput Bayesian Optimisation](https://mlsanity.com/paper/2301.10123/) (2023)
- [Local Bayesian optimization via maximizing probability of descent](https://mlsanity.com/paper/2210.11662/) (2022)
- [A survey and benchmark of high-dimensional Bayesian optimization of discrete sequences](https://mlsanity.com/paper/2406.04739/) (2024)
- [Comparison of High-Dimensional Bayesian Optimization Algorithms on BBOB](https://mlsanity.com/paper/2303.00890/) (2023)
- [Fast Bayesian Optimization of Needle-in-a-Haystack Problems using Zooming Memory-Based Initialization (ZoMBI)](https://mlsanity.com/paper/2208.13771/) (2022)
- [Trieste: Efficiently Exploring The Depths of Black-box Functions with TensorFlow](https://mlsanity.com/paper/2302.08436/) (2023)
- [Revisiting Bayesian Optimization in the light of the COCO benchmark](https://mlsanity.com/paper/2103.16649/) (2021)
- [High-dimensional mixed-categorical Gaussian processes with application to multidisciplinary design optimization for a green aircraft](https://mlsanity.com/paper/2311.06130/) (2023)
- [HypBO: Accelerating Black-Box Scientific Experiments Using Experts' Hypotheses](https://mlsanity.com/paper/2308.11787/) (2023)
- [Mastering the exploration-exploitation trade-off in Bayesian Optimization](https://mlsanity.com/paper/2305.08624/) (2023)
- [Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search](https://mlsanity.com/paper/2206.13130/) (2022)
- [Bayesian Optimization with Informative Covariance](https://mlsanity.com/paper/2208.02704/) (2022)
- [LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions](https://mlsanity.com/paper/2311.11328/) (2023)
- [High-Dimensional Bayesian Optimization Using Both Random and Supervised Embeddings](https://mlsanity.com/paper/2502.00854/) (2025)
- [Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?](https://mlsanity.com/paper/2604.09417/) (2026)
- [A Principle for Global Optimization with Gradients](https://mlsanity.com/paper/2308.09556/) (2023)
- [Approximate Nash Equilibrium Learning for n-Player Markov Games in Dynamic Pricing](https://mlsanity.com/paper/2207.06492/) (2022)
- [SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality](https://mlsanity.com/paper/2406.12661/) (2024)
- [Multi-Variable Batch Bayesian Optimization in Materials Research: Synthetic Data Analysis of Noise Sensitivity and Problem Landscape Effects](https://mlsanity.com/paper/2504.03943/) (2025)
- [Adaptive Replication Strategies in Trust-Region-Based Bayesian Optimization of Stochastic Functions](https://mlsanity.com/paper/2504.20527/) (2025)
- [GimmBO: Interactive Generative Image Model Merging via Bayesian Optimization](https://mlsanity.com/paper/2601.18585/) (2026)
- [LAGO: A Local-Global Optimization Framework Combining Trust Region Methods and Bayesian Optimization](https://mlsanity.com/paper/2603.02970/) (2026)
- [Local Constrained Bayesian Optimization](https://mlsanity.com/paper/2603.07965/) (2026)
- [B3O: Scalable Boltzmann Batch Bayesian Optimization](https://mlsanity.com/paper/2606.30228/) (2026)
- [Efficient multidisciplinary design via Bayesian optimization](https://mlsanity.com/paper/2607.22560/) (2026)
- [GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization](https://mlsanity.com/paper/2608.25116/) (2026)
- [Converge Then Diversify: Decoupling Convergence and Diversity in Multi-Objective Bayesian Optimisation](https://mlsanity.com/paper/2609.13396/) (2026)
- [MF-SCBO : Multi-fidelity Scalable Constrained Bayesian Optimization](https://mlsanity.com/paper/2609.29941/) (2026)
- [STREGO: A Scalable Implementation of a Trust-Region Framework for Noisy Bayesian Optimization](https://mlsanity.com/paper/2610.14338/) (2026)

## Abstract (arXiv)

Efficient Global Optimization (EGO) is the canonical form of Bayesian optimization that has been successfully applied to solve global optimization of expensive-to-evaluate black-box problems. However, EGO struggles to scale with dimension, and offers limited theoretical guarantees. In this work, a trust-region framework for EGO (TREGO) is proposed and analyzed. TREGO alternates between regular EGO steps and local steps within a trust region. By following a classical scheme for the trust region (based on a sufficient decrease condition), the proposed algorithm enjoys global convergence properties, while departing from EGO only for a subset of optimization steps. Using extensive numerical experiments based on the well-known COCO {bound constrained problems}, we first analyze the sensitivity of TREGO to its own parameters, then show that the resulting algorithm is consistently outperforming EGO and getting competitive with other state-of-the-art black-box optimization methods.