# Alpha Entropy Search for New Information-based Bayesian Optimization

arXiv: https://arxiv.org/abs/2411.16586 · HTML: https://mlsanity.com/paper/2411.16586/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Builds on (most cited first, 16 of 16)

- [Max-value Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/1703.01968/) (2017)
- [Predictive Entropy Search for Efficient Global Optimization of Black-box Functions](https://mlsanity.com/paper/1406.2541/) (2014)
- [Flow Annealed Importance Sampling Bootstrap](https://mlsanity.com/paper/2208.01893/) (2022)
- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [Joint Entropy Search for Maximally-Informed Bayesian Optimization](https://mlsanity.com/paper/2206.04771/) (2022)
- [Predictive Entropy Search for Bayesian Optimization with Unknown Constraints](https://mlsanity.com/paper/1502.05312/) (2015)
- [Joint Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/2210.02905/) (2022)
- [Black-box α-divergence Minimization](https://mlsanity.com/paper/1511.03243/) (2015)
- [Dropout Inference in Bayesian Neural Networks with Alpha-divergences](https://mlsanity.com/paper/1703.02914/) (2017)
- [Fast Information-theoretic Bayesian Optimisation](https://mlsanity.com/paper/1711.00673/) (2017)
- [Generalizing Bayesian Optimization with Decision-theoretic Entropies](https://mlsanity.com/paper/2210.01383/) (2022)
- [Trusted-Maximizers Entropy Search for Efficient Bayesian Optimization](https://mlsanity.com/paper/2107.14465/) (2021)
- [Rectified Max-Value Entropy Search for Bayesian Optimization](https://mlsanity.com/paper/2202.13597/) (2022)
- [Bayesian Optimization with Adaptive Kernels for Robot Control](https://mlsanity.com/paper/2402.07021/) (2024)
- [Adversarial α-divergence Minimization for Bayesian Approximate Inference](https://mlsanity.com/paper/1909.06945/) (2019)

## Built on by (most cited first, 1 of 1)

- [Information-theoretic Bayesian Optimization: Survey and Tutorial](https://mlsanity.com/paper/2502.06789/) (2025)

## Abstract (arXiv)

Bayesian optimization (BO) methods based on information theory have obtained state-of-the-art results in several tasks. These techniques heavily rely on the Kullback-Leibler (KL) divergence to compute the acquisition function. In this work, we introduce a novel information-based class of acquisition functions for BO called Alpha Entropy Search (AES). AES is based on the α-divergence, that generalizes the KL divergence. Iteratively, AES selects the next evaluation point as the one whose associated target value has the highest level of the dependency with respect to the location and associated value of the global maximum of the optimization problem. Dependency is measured in terms of the α-divergence, as an alternative to the KL divergence. Intuitively, this favors the evaluation of the objective function at the most informative points about the global maximum. The α-divergence has a free parameter α, which determines the behavior of the divergence, trading-off evaluating differences between distributions at a single mode, and evaluating differences globally. Therefore, different values of α result in different acquisition functions. AES acquisition lacks a closed-form expression. However, we propose an efficient and accurate approximation using a truncated Gaussian distribution. In practice, the value of α can be chosen by the practitioner, but here we suggest to use a combination of acquisition functions obtained by simultaneously considering a range of values of α. We provide an implementation of AES in BOTorch and we evaluate its performance in both synthetic, benchmark and real-world experiments involving the tuning of the hyper-parameters of a deep neural network. These experiments show that the performance of AES is competitive with respect to other information-based acquisition functions such as JES, MES or PES.